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FOREWORD 


The  Training  and  Simulation  Technical  Area  of  the  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (ARI) 
performs  research  and  development  in  areas  that  include  train¬ 
ing  simulation  with  applicability  to  military  training.  Of 
special  interest  is  research  in  the  area  of  training  device 
design  requirements.  The  Army  requires  guidance  in  the  design 
of  training  devices  and  training  systems  if  it  is  to  provide 
the  most  effective  training  at  the  lowest  costs. 

This  research  product  is  a  collection  of  annotated  ab¬ 
stracts  which  may  serve  two  major  purposes.  They  may  serve 
as  a  comprehensive  introduction  to  the  literature  on  training 
device  application  and  evaluation  and  they  may  also  serve  as 
the  beginning  of  a  computerized  data  base  for  training  device 
and  training  system  information.  Such  a  data  base  must  be  de¬ 
veloped  before  future  efforts  to  develop  decision  support  sys¬ 
tems  for  training  design  guidance  may  be  productive. 

Training  device  and  training  system  design  guidance  will 
facilitate  the  efforts  of  training  device  procurers  such  as 
the  Army  Project  Manager  for  Training  Devices  (PM  TRADE)  and 
also  instructional  systems  developers  in  the  Army  Training 
and  Doctrine  Command  (TRADOC) . 


EDGAR  M.  JOHNSON 
Technical  Director 
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AN  ANNOTATED  BIBLIOGRAPHY  OF  ABSTRACTS  ON  THE  USE  OF 
SIMULATORS  IN  TECHNICAL  TRAINING 


EXECUTIVE  SUMMARY 


Requirements : 

To  compile  a  comprehensive  annotated  bibliography  on  the 
use  of  simulators  in  technical  training  and  provide  a  reference 
and  resource  for  researchers,  expert  system  data  base  develop¬ 
ers,  training  program  administrators,  and  simulator  decision¬ 
makers  by  (1)  designing  standardized . formats  for  effectively 
summarizing  the  most  important  and  useful  information  contained 
within  technical  reports  and  other  research  documents,  and 
(2)  writing  annotated  abstracts,  implementing  this  format,  of 
selected  documents  from  the  simulator  training  literature. 


Procedure : 

This  document  primarily  consists  of  149  annotated  abstracts 
of  selected  research  papers  from  the  literature  on  simulator 
training.  The  abstracts  are  arranged  alphabetically  by  first 
author's  name  within  two  broad  groupings:  theoretical  and  em¬ 
pirical.  Each  abstract  is  organized  according  to  the  same  9- 
point  format,  under  the  following  headings:  (1)  Authors; 

(2)  Title;  (3)  Source;  (4)  Topic  Keywords;  (5)  Short  summary; 

(6)  Devices  discussed  or  studied;  (7)  Institution;  (8)  Type  of 
Article;  and  (9)  Abstract.  These  9  headings  are  further  broken 
down  into  subheadings  and  filled  in  with  appropriate  informa¬ 
tion,  if  available  in  the  original  document. 


Findings : 

The  simulator  literature  is  characterized  by  considerable 
diversity — in  terms  of  purpose,  method,  and  theoretical  frame¬ 
work — as  well  as  variability — in  terms  of  quality  and  validity. 
Due  to  small  sample  sizes,  the  absence  of  controls,  or  the  nar¬ 
row  range  of  treatments,  many  of  the  studies  in  the  simulator 
literature  would  be  considered  flawed  by  academic  standards. 
More  effort  needs  to  be  devoted  to  critically  evaluating  and 
consolidating  the  results  of  past  research  in  this  field.  In 
order  to  make  practical  use  of  the  diverse  findings  of  the 
hundreds  of  researchers  who  have  contributed  to  the  simulator 
training  literature,  it  is  desirable  that  these  findings  be 
summarized  and  organized  into  a  single  data  base,  to  allow  for 
critical  comparison,  cross-referencing,  and  ultimately,  assimi¬ 
lation  into  an  expert  system  for  training  design  guidance.  The 
completion  of  these  annotated  abstracts,  organized  by  a 


vii 


standardized  9-point  system,  represents  a  step  toward  the  goal 
of  unifying  the  simulator  literature  and  putting  its  findings 
to  practical  use. 


Utilization  of  Findings: 

These  annotated  abstracts  may  be  used  by  those  interested 
in  or  responsible  for  research  on  simulator  training.  This 
document  may  serve  as  a  reference  guide,  enabling  the  investi¬ 
gator  to  gain  immediate  access  to  a  pertinent  segment  of  the 
simulator  training  literature.  The  document  may  also  be  use¬ 
ful  to  those  responsible  for  planning  future  research  on  train¬ 
ing  system  issues.  The  abstracts  themselves  can  be  entered 
into  a  computerized  data  base  and  information  may  be  accessed 
from  any  of  the  9-point  categories  according  to  a  simple  cod¬ 
ing  scheme.  The  information  contained  in  these  abstracts  may 
therefore  be  helpful  to  those  involved  in  the  development  of 
decision  support  systems  for  training  device  design  guidance. 
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I.  INTRODUCTION 


Those  responsible  for  designing  training  devices  and 
programs  need  a  basis  upon  which  to  decide  what  level  of 
simulator  fidelity  will  produce  the  optimal  training  effect 
for  the  cost.  A  logical  basis  for  making  simulator  fidelity 
decisions  is  the  empirical  findings  of  researchers  who  have 
systematically  studied  the  relationship  between  simulator 
fidelity  and  training  effectiveness.  Since  there  is  already 
a  substantial  body  of  research  literature  in  existence  that 
bears  on  simulator  fidelity,  it  makes  sense  to  try  to  pull 
this  literature  together  to  organize  it,  summarize  it,  and 
make  it  available  to  decision-makers  and  others  who  are 
working  in  the  simulator  field.  An  annotated  bibliography 
can  help  accomplish  this. 

However,  the  abstracts  contained  in  most  annotated 
bibliographies  are  too  brief  and  superficial  to  be  very 
helpful.  Ideally,  an  abstract  should  summarize  the  contents 
of  a  document,  and  not  merely  describe  or  identify  those 
contents.  Previous  abstract  writers  have  tended  to  avoid 
details  altogether,  rather  than  face  the  difficult  task  of 
selecting  only  the  most  essential  details  from  a  given 
document.  An  attempt  has  been  made  here  to  not  only 
summarize  a  document  in  general  terms,  but  also  to  extract 
from  each  document  the  most  significant  details  contained 
within  it,  including  the  key  empirical  findings  from  the 
experimental  studies.  It  is  hoped  that  this  will  help 
provide  a  basis  for  comparing,  analyzing,  and  critically 
evaluating  the  results  of  different  studies  of  different 
devices  at  different  levels  of  fidelity  that  have  been 
conducted  over  the  years.  The  publication  of  these 
abstracts  will  be  a  step  toward  the  establishment  of  a 
master  data  base  containing  the  empirical  results  of  all 
major  studies  bearing  on  simulator  fidelity. 

This  document  is  the  first  iteration  in  a  long  process 
designed  to  accumulate,  organize,  summarize  and  make 
available  the  empirical  and  theoretical  information  the  U.S. 
Army  needs  to  make  training  decisions.  The  format  may 
change  as  new  methods  are  found  for  organizing  and 
summarizing  the  research  literature.  In  addition 
reliability  checks  are  continuing.  While  every  effort  has 
been  made  to  ensure  consistency  and  accurateness  in  the 
annotation  process,  the  short  summaries  and  the  abstract 
summaries,  are  somewhat  subjective.  When  greater  clarity  or 
detail  is  required  the  original  source  citation  is  available 
in  the  reference  section. 
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II  SOURCES,  ORGANIZATION,  AND  PURPOSE  OF  THE  ABSTRACTS 

The  literature  from  which  these  annotated  abstracts 
were  written  was  gleaned,  for  the  most  part,  from  online 
searches  of  the  following  databases:  Psychological 
Abstracts,  NT IS,  ERIC,  GPOM  and  STAR.  The  abstracts  are 
arranged  alphabetically  according  to  the  name  of  the  first 
author  within  two  main  categories:  theoretical  and  review 
articles  and  articles  reporting  empirical  studies.  The 
abstracts  cover  a  period  of  time  from  1957  until  1982.  The 
articles  which  have  been  abstracted  in  this  document  were 
purposely  restricted  to  those  that  directly  relate  to 
simulation  based  training  devices.  This  narrow  scope  was 
chosen  because  resource  constraints  limited  the  total  number 
of  articles  that  could  be  abstracted.  Furthermore,  by 
limiting  the  types  of  articles  abstracted,  it  was  felt  that 
a  more  thorough  job  could  be  done  on  the  articles  chosen. 

Many  of  the  articles  abstracted  here  are  military 
technical  reports,  published  by  such  agencies  as  the  Air 
Force  Human  Resources  Laboratory,  the  Naval  Training  Device 
Center,  and  The  U.S.  Army  Research  Institute.  Copies  of 
these  documents  can  usually  be  ordered  from  the  National 
Technical  Information  Service  (NTIS)  in  Springfield, 
Virginia. 

This  document  contains  abstracts  of  149  articles.  The 
articles  have  been  organized  into  two  main  categories: 
empirical  and  theoretical.  The  abstracts  of  83  empirical 
articles  are  presented  first  and  organized  alphabetically  by 
first  author  or  sponsoring  organization,  if  no  authors  are 
cited.  Within  the  category  of  empirical  articles,  five 
subcategories  are  detailed:  (1)  experiments  -  studies  which 
manipulate  variables,  (2)  surveys  -  studies  which 
systematically  accumulate  data  from  relevant  sources  with 
interviews  or  questionnaires,  (3)  case  studies  -  in  depth 
investigations  of  single  instances,  (4)  analytical  - 
systematic  application  of  an  analysis  methodology  without 
the  manipulation  of  variables,  and  (5)  meta-analyses  - 
statistical  compilations  of  experimental  results  on  specific 
variables  and  relationships  across  several  studies.  The 
abstracts  of  66  theoretical  articles,  also  organized 
alphabetically  by  first  author  or  sponsoring  organization, 
are  presented  next.  Within  the  category  of  theoretical 
articles,  four  subcategories  are  detailed:  (1)  theoretical 
-  a  rigorous  logical  approach  to  issues,  (2)  review  -  a 
summary  of  some  portion  of  the  research  literature,  (3) 
conceptual  -  opinions,  statements  of  "important"  issues, 
etc.,  and  (4)  methodological  -  plans,  approaches,  and 
research  guidelines.  Finally,  a  reference  list  of  all  the 
articles  abstracted  is  presented.  This  reference  list 
contains  the  page  number  of  the  corresponding  abstract. 


2 


I 


c.  Tests  or  Trials/Timing:  2  post-tests 

d.  Number  of  Different  Types  of  Measures:  1 


e.  Description  of  Measurements  and  Ratings:  Lateral 
deviation  of  wheel  rim  from  true  (mean  peak) 


f.  Experimental  Setting/Training  Context:  Laboratory 
experimental,  hands-on  (where  device  functional) 


g.  Statistical  Methods:  t-test,  one-way  repeated 
measures  ANOVA;  two-way  repeated  measures  ANOVA; 
Pearson  correlations. 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 

(2)  Fidelity  Levels: 


Physical 


Functional 


High 

Low 


Actual  equipment 
working 

Computer  graphics 
Degraded  model  of 

actual  equipment  Medium 
Actual  equipment 
not  working 
Line  drawings 


High 

High 


Medium 


High 

Low 


Low 

Low 


(3)  Type  of  Task/Skill  Required:  truing  bicycle 
wheel;  psychomotor;  motor;  perceptual. 

(4)  Task  Difficulty:  Medium-low 


i.  Stage  of  Training:  introduction;  familiarization; 
skill 


j.  Trainee  Sophistication:  novice 


k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 


User  Acceptance  or  Attitude:  not  discussed 


m.  Use  of  Instructional  Features: 

(1)  Intensity:  minimal  to  incomplete  (for 
computer-graphics  device) 

(2)  Features  used:  not  specified 


9.  Abstract: 


a.  Study  Synopsis:  This  paper  described  the 
experimental  implementation  of  a  design  formulated  in 
ar  earlier  part  of  this  study  (Baum,  et.  al . , 


%  » 
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1.  Authors:  Baum,  David  R. ,  Riedel,  Sharon, 

Hays,  Robert  T. ,  &  Mirabella,  Angelo 

2.  Title:  Training  Effectiveness  as  a  Function  of  Training 
Device  Fidelity. 

3.  Source:  Simtrain  Task  2  Final  Report,  August  1982. 

4.  Topic  Keywords:  Fidelity  ;  Physical  Fidelity  ; 
Functional  Fidelity  ;  Fidelity  Specifications  ; 

Maintenance  Training  Simulation  ;  Perceptual-Motor  tasks  . 

5.  Short  Summary:  This  experiment  examined  the  effects  of 
varying  training  device  fidelity  on  training  effectiveness 
in  a  perceptual-motor  task.  The  experiment  found  that 
significant  and  meaningful  effects  occur  in  relation  to  the 
physical  dimension  of  fidelity  (device  appearance),  but  not 
in  relation  to  functional  fidelity  (device  operability  and 
response) .  The  experiment  had  problems  in  the  experimental 
methodology,  however. 

6.  Devices:  Bicycle-wheel  truing  devices 

a.  Actual  Equipment  in  working  order 

b.  Computer  graphics  display  of  equipment 

c.  Degraded  3-dimensional  model  of  equipment 

a.  Actual  equipment  but  not  working  (none  of  parts 
moved) 

e.  Line  drawings  of  equipment 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (PERI-IC),  5001 
Eisenhower  Avenue,  Alexandria,  VA  22333. 

b.  Performing  Organization:  Honeywell  Systems  and 
Research  Center,  2600  Ridgway  Parkway,  P.O.  Box  312, 
Minneapolis,  MN  55440. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  5 

b.  Description  of  Groups:  ( 1 ) — ( 5 )  Subjects:  (Each 

group)  17  male  and  3  female,  3  of  whom  were  technical 
school  students,  mean  age  approximately  17, 
non-college-bound. 


Under  forward  chaining,  poor  performance  often  results 
in  the  termination  of  the  student  prior  to  completing 
the  task.  In  the  case  of  a  weapons  delivery  task  such 
as  this,  the  student  under  the  backward  chaining  method 
has  the  opportunity  to  drop  more  bombs  per  unit  of 
training  time  than  would  a  student  under  the  whole  task 
method  or  a  forward  chaining  method.  Given  that 
bomb-dropping  is  the  essence  of  this  particular  task, 
backward  chaining  allows  more  actual  practice  time  than 
does  either  the  whole  task  method  or  forward  chaining. 
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different  in  initial  skill  on  the  task.  All  training 
and  testing  were  done  in  the  Advanced  Simulator  for 
Pilot  Training. 

The  backward-chaining  group  learned  the  bombing  tasks 
as  four  sub-tasks  in  the  following  order:  (1)  Final; 

(2)  Roll-in;  (3)  Base  Leg;  (4)  Down-Wind.  After 
learning  each  sub-task  to  criterion,  they  "graduated" 
to  learning  the  task  which  was  previous  to  it  in 
real-life  terms.  After  learning  all  the  sub-tasks  to 
criterion,  they  performed  five  trials  of  the  whole 
task,  with  all  maneuvers  in  their  proper  sequence. 

These  last  five  trials  constituted  their  post-test. 

The  whole-task  group  performed  30  trials  of  the  whole 
task,  in  which  no  segment  was  distinguished  for  them  as 
a  "test".  Their  performance  of  trials  15-19  were 
subsequently  selected  as  the  post-test  to  be  compared 
with  the  backward  chaining  group.  The  reason  given  was 
that  by  trial  14  this  group  had  received  training  time 
equal  to  what  the  backward-chained  group  had  received 
by  the  criterion  method. 

b.  Results: 

(1)  Key  Data: 

(a)  Mean  Circular  Error  (over  5  trials) : 

Backward  Chaining  Group  :  134.38  feet 

Whole  Task  Group:  182  .34  feet 

(b)  Percent  Reaching  Criterion  in  equal 
time: 

Backward  Chaining  Group:  7  0% 

Whole  task  group:  30% 

(2)  Verbal  Description  of  Results: 

Backward-chained  subjects  showed  significantly 
greater  accuracy  in  the  task  after  equal  training 
time.  (Chi  Square  *  5.0,  df=l,  p<.05).  Not  only 
was  the  accuracy  of  the  subjects  in  the  backward 
chaining  condition  superior  when  training  time  for 
the  two  groups  was  equated,  but  the  rate  at  which 
students  reached  criterion  was  significantly 
faster  under  the  backward  chaining  condition. 

c.  Authors'  Conclusions:  The  results  of  the  present 
study  indicate  a  clear,  systematic  reduction  in 
circular  error  with  the  use  of  a  backward  chaining 
method  for  introducing  the  30  degree  dive  bomb  task. 
However,  the  present  study  does  not  address  the 
potential  effectiveness  of  forward  chaining  as  an 
alternative  to  the  whole  task  approach.  Both  chaining 
methods  offer  some  advantages  over  traditional  whole 
task  methods. 
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hands-on 


g.  Statistical  Methods:  groups  mean,  group 
percentages 

h.  Variables  being  manipulated: 

(1)  Training  Device:  Advanced  Simulator  for  Pilot 
Training,  G-seat  inflated  but  not  operating, 
platform  motion  not  operating 

(2)  Estimated  Fidelity  Levels: 

(a)  Physical:  high 

(b)  Functional:  high 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  part-task,  whole-task 

(4)  Task  Difficulty:  high 

i.  Stage  of  training:  skill, 
familiarization/introduction  new  task 

j.  Trainee  sophistication:  generally 
high/ intermediate/e xpert 

k.  Incorporation  of  device  into  P.O.I.:  not 
applicable 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified,  presumed  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  Training  by  "backwards  chaining" 
involves  separating  the  task  to  be  learned  into  steps, 
which  are  then  learned  in  reverse  order  of  their  real¬ 
lifetime  sequence.  Behavioral  principles  indicate  that 
this  technique  should  be  superior  to  the  whole-task, 
continuous,  beginning-to-end  training  method  for  many 
types  of  tasks.  In  this  study,  the  backwards  chaining 
method  was  applied  in  teaching  a  simulated  dive-bombing 
task  to  experienced  pilots.  Their  training  time  to 
criterion  in  the  task,  and  their  accuracy  in  test  runs 
of  the  whole  task,  as  measured  by  circular  error 
(distance  of  simulated  bomb  impact  from  target)  were 
compared  with  the  performances  of  an  equivalent  group 
trained  by  the  whole-task  method. 

The  subjects  were  two  groups  totalling  20  Air  Force 
Instructor  pilots  who  were  familiar  with  the  simulated 
aircraft  but  not  with  the  task  to  be  learned.  By 
pre-test  the  groups  were  found  not  to  be  significantly 
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1.  Authors:  Bailey,  John,  Hughes,  Ronald,  &  Jones,  William 
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2.  Title:  Application  of  backward  chaining  to 
air-to-surf ace  weapons  delivery  training. 

3.  Source:  Air  Force  Human  Resources  Laboratory 
Technical  Report  (AFHRL-TR-79-63) ,  April  1980. 

4.  Topic  Keywords:  Flight  Training  ;  Backward  Chaining  ; 
Whole-task  ;  High  Fidelity  Simulator  ;  Response  Chaining  ; 
Weapons  Delivery  ;  Air-to-ground  attack  . 

5.  Short  Summary:  Two  groups  of  student  pilots  were 
trained  on  a  dive-bombing  task,  one  group  by  the  whole-task 
method,  and  the  other  by  a  backward-chaining  method.  Both 
groups  were  trained  on  the  simulation  equipment.  The 
backward-chaining  group  reached  criterion  with  fewer  trials, 
and  were  more  accurate  given  an  equal  number  of  trials. 

6.  Device:  Advanced  Simulator  for  Pilot  Training, 
high-fidelity  visual  and  instrument  flight  simulator 

7.  Institutions: 

a.  Sponsor:  AFHRL  HQ  Air  Force  Human  Resources 
Laboratory  (AFSC) ,  Brooks  AFB ,  TX  78235. 

b.  Performing  organization:  Flying  Training  Division, 
AFHRL,  Williams  AFB,  AZ  85224. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects  in  Backward  Chaining  Group: 
approximately  10  Air  Force  Instructor  pilots 

(2)  Subjects  in  Whole-task  Group:  same 

c.  Tests  or  Trials: 

(1)  For  Backward  Chaining  Group:  one  post-test  (5 
trials) 

(2)  For  Whole-task  Group:  5  in-process  trials 

(3)  For  Both  Groups:  pretest  to  establish 
baseline 

d.  Number  of  Different  Types  of  Measures  Used:  1 

e.  Description  of  Measures/Ratings:  Mean  circular 
error  (distance  of  simulated  bomb  impact  from  target) 

f.  Experimental  Setting/Training  Context:  Laboratory, 
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"Note:  Many  of  the  above  categories  require  estimates  or 

subjective  judgments  on  the  part  of  the  reviewer.  Some 
errors  and  inaccuracies  are  inevitable.  All  such  errors  and 
inaccuracies  are  regrettable  and  every  reasonable  effort  has 
been  made  to  avoid  them. 


IV  EXPLANATION  OF  THE  ABSTRACT  FORMAT  FOR  THEORETICAL 
ARTICLES 


Each  of  the  annotated  abstracts  of  theoretical  articles  is 
organized  under  the  following  9-point  format: 

1.  Author  Is)  :  The  author (s)  of  the  document,  starting  with 
the  last  name  of  the  first  author;  if  the  authors  are  not 
identified,  the  performing  organization  is  identified  in  the 
author's  place. 

2.  Title:  The  title  of  the  document,  including  subtitles. 

3.  Source :  The  publication  source  of  the  document, 
including,  if  available,  the  identifying  code  numbers  of  the 
sponsoring  agency's  catalog  system. 

4.  Topic  Keywords :  Keywords,  such  as  those  appearing  as 
topic  headings  in  major  relevant  reference  book 
indexes--e.g.  "Training  Device  Effectiveness"  or  "Flight 
Simulation" — which  identify  the  specific  subject  matter  of 
the  document. 

5.  Short  Summary:  A  short  summary  of  the  contents  of  the 
document  for  quick  feference  or  review. 

6.  Device ( s)  :  The  devices  discussed  in  the  article. 

7 .  Institutions 

a.  Sponsor :  The  agency  which  sponsored,  contracted, 
funded,  directed,  or  arranged  for  the  effort. 

b.  Performing  Organization:  The  agency  which 
conducted  the  effort,  which  employed  the  authors  of  the 
article,  or  with  which  the  authors  of  the  atricle  were 
affiliated. 

8.  Type  j^f.  Article :  (e.g.  Review,  conceptual, 

theoretical,  methodological) 

9.  Abstract :  This  section  contains  a  verbal  summary  of  the 
document  with  the  authors'  conclusions. 
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(2)  Features  Used :  A  list  of  instructional 
features  of  the  device,  as  specified  by  the 
authors,  that  are  actually  used  in  the  training 
program  under  study — e.g.  freeze  capability, 
restart/resequence  capability,  sign-in  capability, 
augmented  feedback,  record/playback,  adaptive 
syllabus,  etc. 

9.  Abstract :  This  section  contains  the  verbal  summary  of 

the  document  as  well  as  the  summary  statistics. 


a.  Study  Synopsi s ;  A  verbal  summary  of  the  study,  its 
purpose,  hypotheses,  methods,  procedures,  subjects, 
controls,  etc. 


b.  Results ;  Here  is  where  the  most  significant  or 
representative  empirical  data  extracted  from  the 
document  are  presented. 

(1)  Key  Data ;  The  most  significant  or 
representative  statistics  are  presented  in  tabular 
form,  whenever  possible. 

(2)  Verbal  Description :  A  discussion  of  the  key 
data  and  their  implications. 

c.  Authors1  Conclusions :  The  authors'  interpretations 
of  their  findings  and  conclusions. 


*  Note:  Many  of  the  above  categories  require  estimates  of 
subjective  judgments  on  the  part  of  the  author  of  the 
abstract.  Some  errors  and  inaccuracies  are  inevitable.  All 
such  errors  and  inaccuracies  are  regrettable  and  every 
reasonable  effort  has  been  made  to  avoid  them. 
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(3)  Type  Task/Skill  Required:  Classification 
of  the  type  of  behavioral  skills  required  in 
performance  of  the  task  being  tr ained--e . g. 
operations,  cognitive,  psychomotor,  perceptual, 
procedural,  etc. 

(4)  Task  Difficulty :  Classification  of  the  level 
of  difficulty  of  the  task  being  trained — i.e. 
low/medium/high. 


i.  Stage  of  Training:  The  stage  of  training  at  which 
the  simulator  is  used  in  the  training  program--e.g. 
nomenclature,  familiarization,  procedural,  skill, 
transition,  refresher,  proficiency  testing,  etc. 


j.  Trainee  Sophistication:  The  level  of 
sophistication  of  the  trainees  in  the  study — i.e. 
novice,  intermediate,  advanced,  expert,  graduates  of 
the  program,  2  years  on  the  job,  etc. 


k.  Incorporation  qL  Device  into  Program  £f. 
Instruction:  The  instructional  method  by  which  the 

training  device  is  employed  when  it  is  incorporated 
into  a  training  program — e.g.  self-paced,  lock-step, 
instructor-managed,  etc. 


1.  User  Acceptance  2jl  Attitude :  Attitudinal  factors 
can  have  an  impact  on  training  device  effectiveness. 

If  the  authors  provide  any  indication  of  the  attitude 
of  either  instructors  or  students,  it  will  be  included 
here. 

(1)  Instructors :  Indications  of  the  attitude  of 
the  instructors  toward  the  training  device, 
derived  from  questionnaires,  interviews,  etc. 

(2)  Students :  Indications  of  the  attitude  of  the 
students  toward  the  training  device,  derived  from 
questionnaires,  interviews,  etc. 


m.  Use  of  Instructional  Features :  The  utilization  of 
special  instructional  features  is  one  potential 
advantage  of  simulators  over  actual  equipment,  and  may, 
in  some  cases,  compensate  for  lower  fidelity.  The 
presence  and  potential  impact  of  such  features  are 
noted  here. 

(1)  Intensity :  An  estimate  of  the  degree  to  which 
instructional  features  are  used  in  the  training 
program  and  might  conceivably  affect  the 
effectiveness  of  the  device--e.g.  minimal, 
incomplete,  intensive,  etc. 
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(1)  Subj  ects :  The  number  of  experimental  subjects 
and  their  key  demographic  characteristics  as 
described  by  the  authors--e.g.  age,  sex,  level  of 
experience,  etc. 

(2)  Controls :  The  number  of  control  subjects  and 
their  key  demographic  characteristics  as  described 
by  the  authors--e . g.  age,  sex,  level  of 
experience,  etc. 

c.  Tests  or  Trials/Timing:  an  explanation  of  the 
timing  and  sequence  of  the  tests  or  trials  in  relation 
to  the  treatment--e.g.  pretest,  in-process  trial, 
post-test,  etc. 

d.  Number  of  Different  Types  of.  Meas.ux.es :  The  number 
of  types  of  measures  or  dependent  variables  reported  by 
the  authors  and/or  taken  into  consideration  in  reaching 
the  study's  conclusions. 

e.  Description  of.  Measurements  and  Ratings:  A 
description  of  each  type  of  measure,  rating,  or 
dependent  variable  reported  in  the  study — e.g.  number 
of  errors,  time-to-completion,  instructor  ratings,  etc. 

f.  Experimental  Setting/Training  Context: 
Identification  of  the  setting  or  context  of  the  study: 
e.g.  institutional,  classroom,  hands-on,  field, 
on-the-job  training,  etc. 

g.  Statistical  Methods :  The  statistical  method(s) 
specified  by  the  authors;  if  the  authors  do  not 
specify  what  statistical  method  they  employed,  this 
will  be  indicated. 


h.  Variables  Being  Manipulated :  The  independent 
variables  of  the  study. 

(1)  Training  Device l s ) :  Training  devices  that  are 
being  tested,  compared,  or  manipulated  in  the 
study. 

(2)  Fidelity  Level (si:  Estimated  fidelity  level 
of  experimental  device--e.g.  low/medium/high. 

(a)  Physical :  Estimated  level  of  physical 
fidelity  of  the  experimental  device--e.g. 
low/medium/high. 

(b)  Functional :  Estimated  level  of 
functional  fidelity  of  the  experimental 
device--e.o  low/medium/high. 


III.  AN  EXPLANATION  OF  THE  ABSTRACT  FORMAT  FOR  EMPIRICAL 
ARTICLF"’ 


Each  of  the  annotated  abstracts  of  empirical  articles  is 
organized  under  the  following  9-point  format: 

1.  Author ( s )  :  The  author(s)  of  the  document,  starting  with 
the  last  name  of  the  first  author;  if  the  authors  are  not 
identified,  the  performing  organization  is  identified  in  the 
author's  place. 

2.  Title:  The  title  of  the  document,  including  subtitles. 

3.  Source :  The  publication  source  of  the  document, 
including,  if  available,  the  identifying  code  numbers  of  the 
sponsoring  agency's  catalog  system. 

4.  Topic  Keywords :  Keywords,  such  as  those  appearing  as 
topic  headings  in  major  relevant  reference  book 
indexes--e.g.  "Training  Device  Effectiveness"  or  "Flight 
Simulation" — which  identify  the  specific  subject  matter  of 
the  document. 

5.  Short  Summary:  A  SHORT  summary  of  the  contents  of  the 
document  for  quick  reference  or  review. 

6.  Device  ( s) :  The  training  device(s)  being  tested, 
compared,  manipulated,  or  otherwise  employed  in  the  study 
for  experimental  or  evaluative  purposes. 

7 .  Institutions 

a.  Sponsor :  The  agency  which  sponsors,  contracts, 
funds,  directs,  or  arranges  for  the  study. 

b.  Performing  Organization:  The  agency  which 
conducted  the  study,  which  employs  the  authors  of  the 
study,  or  with  which  the  authors  of  the  study  are 
affiliated. 

8.  Type  of.  Article :  (experiment,  survey,  case  study,  meta 
analytic) — where  applicable  the  following  subheadings 
appear : 

a.  Number  of  Groups :  The  number  of  groups  involved  in 
the  experiment. 


b.  Description  oi.  Groups :  A  brief  description  of 
subjects  and  controls. 


The  accumulation  of  these  annotated  abstracts  serves 
two  major  purposes.  First/  the  abstracts,  as  they  now 
stand,  can  be  entered  into  the  literature  on  training  device 
research  and  serve  as  an  excellent  introduction  and  summary 
for  researchers  new  to  this  area.  A  second  and  possibly 
more  important  purpose  for  these  abstracts  is  that  they  will 
be  entered  into  a  computerized  data  base  for  future 
analyses.  The  abstracts  have  been  formatted  to  facilitate 
their  coding  for  easy  computerized  retrieval  of  information. 
The  next  sections  explain  the  format  for  the  abstracts  of 
the  empirical  and  theoretical  articles. 


Specification  of  Training  Simulator  Fidelity:  A 
Research  Plan,  1982),  whose  purpose  was  to  investigate 
the  training  effect  of  varying  simulator  fidelity  in 
military  maintenance  tasks.  In  this  study,  the 
manipulation  of  simulator  fidelity  was  applied  to  a 
perceptual-motor  task  of  moderately  low  difficulty. 

Following  the  definition  of  Hays  described  in  the 
earlier  paper,  the  fidelity  of  simulation  was  separated 
into  two  dimensions--1' physical  "  and  "f  unctional  "--and 
an  experimental  task  was  chosen  in  which  these  two 
dimensions  could  be  varied  independently  of  each  other 
insofar  as  possible.  The  experimental  design, 
suggested  by  Hays  (1981)  was  as  follows  (the  five 
blocks  marked  are  the  ones  investigated  in  this 
experiment) : 


High:  works 
with  effect 


Actual  Device  3-D  Model 


Low 

Pictures/ 

Graphics 


XX  XX 

XX  XX 


Medium:  works  XX 

with  no  effect  XX 


Low:  Does  XX  XX 

not  work  XX  XX 


Results  of  experimentation  in  the  five  blocks  would,  it 
was  expected,  represent  the  impact  of  variance  within 
the  full  range  of  fidelity  in  the  two  dimensions. 

The  task  selected  for  study  was  the  truing  of  a  bicycle 
wheel.  It  matched  the  design  requirements  well,  was 
easily  studied  and  controlled  in  a  laboratory,  and  was 
felt  to  be  representative  of  Army  perceptual-motor 
maintenance  tasks  in  general. 

Subjects  were  chosen  as  being  representative  of  the 
Army  volunteer  population.  There  were  100  non-college- 
bound  men  and  women  with  a  mean  age  of  17.25  years. 

They  were  randomly  assigned  to  five  conditions,  groups 
of  20  each,  with  the  following  constraints:  each 
condition  had  17  males,  3  females  and  3  technical 
students . 
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The  five  experimental  devices  were,  in  brief:  (1) 
Actual  equipment  in  working  order  —  high  physical, 
high  functional  fidelity;  (2)  computer  graphics 
display  of  equipment  —  low  physical,  high  functional 
fidelity;  (3)  degraded  3-dimensionai  model  of 
equipment,  parts  moved  but  produced  no  effect  —  medium 
physical,  medium  functional  fidelity;  (4)  actual 
equipment  but  no  parts  working  —  medium  physical,  low 
functional  fidelity;  and  (5)  line  drawings  of 
equipment  —  low  physical,  low  functional  fidelity. 

Subjects  were  trained  on  the  five  devices  separately, 
and  were  subsequently  given  two  15-minute  performance 
tests  on  the  actual  equipment  (which  was  the  High/High 
training  device) .  Measurements  were  taken  of  the 
lateral  deviation  of  the  wheels  from  "true"  (another 
component  of  "true",  roundness,  was  not  studied) . 

Proposals  for  future  research,  particularly  in  the 
cognitive  area,  were  also  made  in  this  paper. 

b.  Results: 

(1)  Key  Data: 

Average  of  sum  of  peak  rim 
deviations  for  Trials  1  and  2 
(thousandths  of  an  inch) 


Initial 

Conclusion 

GlQUgU. 

Set-up 

of  tests 

Actual  Equipment  Trainer 
Nonoperational  Actual 

275 

85 

equipment 

280 

87 

Computer  Graphics 

277 

135 

Line  Drawings 

265 

140 

(Reviewer's  Note:  The  numbers  above 

are 

approximations  drawn  from  graph  representations  in 
the  text.  (6  measurements  were  actually  taken  for 
each  test,  at  3-minute  intervals).  Text  did  not 
supply  data  on  5th  device.) 

Also,  a  group  of  "experts"  were  tested  on  the 
device,  whose  greatly  superior  performance  to 
subjects  showed  that  a  ceiling  effect  was  not 
taking  place  among  subjects. 

(2)  Verbal  Description:  All  of  the  t-tests 
comparing  the  initial  and  final  measurements  of 
each  group  were  statistically  significant  at 
p- 0.005.  Any  adjustment  made  will  still  yield 
significance  levels  of  at  least  p-. 05.  These 


results  indicate  that  regardless  of  the  training 
device  used,  subjects'  mean  performance  improved 
significantly  over  the  course  of  each  trial. 


A  two-way  repeated  measures  ANOVA  revealed  that 
the  main  effect  of  physical  similarity  is 
statistically  significant,  F(l,  75)  =  4.157, 
pJj.05.  Neither  the  main  effect  of  functional 
similarity  nor  any  of  the  interactions  effects  are 
significant . 

c.  Authors'  Conclusions:  Training  on  all  five 
devices  led  to  significant  improvement  during 
performance  trials.  Even  subjects  trained  using 
line  drawings  of  the  equipment  showed  transfer  of 
learning.  All  of  the  devices  were  therefore 
effective  in  training-wheel  truing. 

The  performance  of  the  expert  group  shows  that  the 
lack  of  a  significant  fidelity  effect  is  not  due 
to  a  ceiling  effect — none  of  the  other  groups 
performed  as  well  as  the  expert  group. 

For  training  of  this  task,  effectiveness  is  a 
function  of  the  physical  similarity  of  the 
training  device  to  the  actual  equipment,  but  is 
not  affected  by  functional  similarity. 

Without  an  optimized  interface  and  training 
method,  the  computer  graphics  device  provides  no 
learning  facilitation  for  this  task  beyond  that 
found  with  a  set  of  line  drawings. 

While  the  general  level  of  fidelity  had  no  effect 
on  training  effectiveness...  it  was  shown  that 
physical  similarity  has  a  significant  impact; 
functional  similarity  has  no  effect.  This  seeming 
paradox,  that  general  fidelity  does  not  achieve 
significance  while  one  of  its  dimensions  does,  is 
attributable  to  the  high  variablility  within  each 
group  and  the  increased  degrees  of  freedom  and 
estimation  precision  that  comes  from  combining 
groups  to  assess  the  effects  of  physical  and 
functional  similarity. 

Fidelity  must  be  operationalized  as  consisting  of 
at  least  two  dimensions  --  physical  and  functional 
similarity. 
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1.  Authors:  Bernstein,  Bernard  R.  &  Gonzalez,  Barbara  K. 

2.  Title:  Learning,  Retention  and  Transfer  in  Military 
Training,  1970. 

3.  Source:  Technical  Report:  NAVTRADEVCEN  69-C-0253-1. 

4.  Topic  Keywords:  Imagery  ;  Task  Difficulty  ; 

Low  Fidelity  . 

5.  Short  Summary:  Three  experiments  address  the  issues  of 
imagery  and  fidelity  in  the  training  of  procedural  tasks. 
These  experiments  validate  the  use  of  low  fidelity  paper  and 
pencil  simulations  and  suggest  that  imagery  facilitates 
learning. 

6 .  Devices : 

a.  Experiment  Three. 

(1)  Communications  console  (the  test  apparatus). 

(2)  Preliminary  training  aid,  cartoon  slides 
analogous  to  console  problems,  plus  text. 

(3)  Preliminary  training  aid,  cartoon  slides 
directly  representing  console  problems,  plus  text. 

(4)  Same  as  (3),  without  text. 

(5)  Verbal  text  only. 

(6)  Training  manual. 

b.  Experiment  Four. 

(1)  Communications  console  (the  test  apparatus) . 

(2)  Preliminary  training  aid,  cartoon  slides 
directly  representing  console  problems,  plus  text. 

(3)  Training  manual. 

c.  Experiment  Five. 

(1)  Communications  console  (the  test  apparatus). 

(2)  High  fidelity  drawing  of  console. 

(3)  Medium  fidelity  drawing  of  console. 

(4)  Low  fidelity  drawing  of  console. 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Device  Center,  Orlando,  FL 

b.  Performing  Organization:  Honeywell  Inc.,  Systems 
and  Research  Division,  St.  Paul,  MN  55113 

8.  Type  of  Article:  Experiments. 

a.  Number  of  Groups:  Experiment  III,  5;  Experiment 
IV,  3;  Experiment  V,  3 


b.  Description  of  Groups: 

(1)  Experiment  III:  (l)-(5),  four  experimental 
and  one  control  group;  each  8  volunteer  college 
undergraduates,  age  18-25. 

(2)  Experiment  IV:  (l)-(3),  two  experimental  and 
one  control  group;  each  10  volunteers  from 
college  introductory  psychology  classes. 

(3)  Experiment  V:  (l)-(3),  each  group  16 
volunteer  male  undergraduate  college  students. 

c.  Tests  or  trials/timing:  In-process  trials, 
post-tests,  delayed  post-tests. 

d.  Number  of  different  types  of  measures:  2. 

e.  Description  of  measurements  and  ratings: 

(1)  Response  time  to  complete  sub-routines. 

(2)  Accuracy. 

f.  Experimental  setting/training  context:  Laboratory, 
classroom  and  hands-on. 

g.  Statistical  methods:  t-tests  and  ANOVA. 

h.  Variables  being  manipulated: 

(1)  Devices:  as  in  section  6  above. 

(2)  Fidelity  levels: 


Physical 

Functional 

Experiment  III, 
all  devices 

Low 

Low 

Experiment  IV, 
all  devices 

Low 

Low 

Experiment  V, 
device  b 

Medium-Low 

Low 

&  Very 

Low 

device  c 

Low 

Low 

&  Very 

Low 

device  d 

Very  Low 

Low 

&  Very 

Low 

(3)  Type  of  task/skill  required:  cognitive, 
psychomotor,  procedural,  part-task. 

(4)  Task  difficulty:  medium. 


i.  Stage  of  training:  introduction. 

j.  Trainee  sophistication:  novice. 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 
(not  really  applicable). 

l.  User  acceptance  or  attitude:  not  discussed. 
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m.  Use  of  instructional  features: 

(1)  Intensity:  incomplete. 

(2)  Features  used:  not  specified. 

9.  Abstract: 

a.  Study  Synopsis:  This  study  continued  an 
investigation  into  factors  affecting  training 
effectiveness,  in  which  two  previous  experiments  had 
already  been  conducted;  theoretical  commentary  on 
those  experiments  was  included  here.  Those  two 
experiments  had  generated  tentative  hypotheses  relevant 
to  the  roles  of  imagery,  device  fidelity,  and  cognitive 
understanding  in  the  learning  of  procedural  tasks,  and 
the  three  experiments  reported  here  tested  and  expanded 
on  those  hypotheses.  Another  part  of  this  study  was  a 
survey  of  training  personnel  regarding  the  issue  of 
task  difficulty:  the  results  of  the  survey  were 
inconclusive  and  are  not  discussed  below. 

b.  Experiment  III:  The  first  experiment  of  the  later 
phase  of  the  overall  investigation  explored  the  use  of 
imagery  materials  as  aids  to  enhance  verbal  material 
which  introduced  trainees  to  tasks  on  a  communications 
console.  In  preliminary  training  (prior  to 
"acquisition  training  on  the  console  itself,  which  was 
followed  by  transfer  testing  on  the  console) ,  four 
groups  of  trainees  were  given  aids  in  the  form  of 
cartoon  drawings  with  different  degrees  of  relevance  to 
the  tasks  to  be  performed;  one  of  these  groups  was 
deprived  of  any  text;  another  group  heard  the  text  but 
was  deprived  of  drawings;  the  fifth  control  group,  was 
simply  given  a  training  manual  to  study.  Measurements 
of  performance  were  time  to  complete  the  task,  and 
accuracy . 

Results  of  Experiment  III:  ANOVA's  were  performed  on 
the  acquisition  and  transfer  data  for  both  response 
time  and  accuracy  scores.  The  various  conditions 
failed  to  produce  a  significant  main  effect  in  either 
session  for  either  measure.  The  trial  blocks  effect 
was  reliable,  showing  a  learning  effect  for  both  time 
and  accuracy  in  both  training  and  transfer  sessions. 

Also,  perhaps  most  importantly,  it  was  discovered  that 
the  control  group  in  the  experiment  performed 
significantly  better  than  the  control  group  in  an 
earlier  experiment  for  the  same  task.  This  could  be 
attibuted  to  improvements  in  the  instruction  methods 
between  the  two  experiments,  and  it  would  account  for 
the  lack  of  a  significant  difference  between  the 
experimental  and  control  groups  in  this  latter 
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experiment . 

c.  Experiment  IV:  This  expanded  on  Experiment  III  by 
analyzing  the  effect  of  trainees'  verbal  aptitude,  as 
measured  by  the  Ammons  Quick  Test,  on  performance.  An 
interaction  was  hypothesized  between  verbal  aptitude 
and  the  impact  of  imagery  technique  on  learned 
performance.  The  number  of  conditions  was  reduced  from 
five  to  three  (one  control,  two  experimental) . 

Results  of  Experiment  IV:  Neither  the  verbal  skill 
levels  nor  the  imagery  conditions  produced  a 
statistically  significant  main  effect.  The  elevated 
performance  of  the  control  group  vis-a-vis  the  earlier 
"Experiment  II"  control  group,  again  appeared  to 
account  for  the  lack  of  differences  between  groups  as 
an  effect  of  imagery. 

d.  Experiment  V:  This  experiment  investigated  the 
effect  of  six  different  levels  of  simulator  fidelity. 
The  task  was  the  same  procedural  task  as  in  Experiments 
III  and  IV.  The  training  aids  were  line  drawings  of 
the  console.  Physical  fidelity  was  varied  by  the 
similarity  of  the  drawings  to  the  console.  Two  levels 
of  functional,  or  "response"  fidelity  were  employed: 
low  (written  responses  on  the  drawings,  rather  than 
instrument  manipulations)  and  very  low  (verbal 
responses,  which  of  course  required  no  manual 
activity) . 

Results  of  Experiment  V:  Neither  variation  in  physical 
("stimulus")  or  functional  ("response")  fidelity 
produced  a  significant  main  effect. 

e.  Authors'  Conclusions:  The  research  accomplished 
has  dealt  with  only  one  category  of  behavior.  The 
findings  for  skills  other  than  procedural  may  differ. 

Of  the  variables  under  study,  the  findings  associated 
with  fidelity  of  simulation  appear  to  be  the  most 
conclusive.  We  now  have  considerable  evidence  that 
relatively  high  levels  of  training  effectivenes  for 
procedural,  tasks  can  be  maintained  in  the  absence  of 
high  physical  fidelity  between  training  and  transfer 
tasks.  This  appears  to  be  true  both  in  terms  of  the 
stimulus  and  respose  components  of  the  task  ("Physical" 
and  "Funcional"  f idelity--reviewer ' s  note).  These 
findings  lend  support  to  Wittrock's  concept  of  mediated 
generalization  as  the  basis  of  transfer  in  procedural 
skills.  Apparently,  subjects  can  cope  with  variations 
in  the  stimulus  and  response  components  of  the  task  as 
long  as  the  basic  system  structure  remains  unchanged. 
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With  respect  to  an  interaction  between  trainee  verbal 
skill  and  imagining  techniques,  the  present 
investigation  does  not  appear  to  provide  conclusive 
results.  The  absence  of  predictive  validity  for  the 
Ammons  Quick  Test  may  be  the  result  of  an  excessively 
restricted  range  in  verbal  skills  for  the  subjects 
used.  Over  half  of  the  subjects  scored  at  the  mean  for 
college  students  or  within  two  points  of  the  mean.  It 
may  be  worthwhile  to  run  another  group  of  subjects  who 
score  significantly  lower  than  the  college  students 
used  in  this  study. 

Regarding  the  effects  of  imagery  in  teaching  procedural 
tasks,  these  experiments  are  inconclusive,  especially 
as  there  are  several  factors  which  may  influence  the 
effectiveness  of  imagery,  quite  apart  from  the  use  of 
imagery  per  se.  Among  the  more  important  are:  (1) 
subject  verbal  skill,  unanswered  by  Experiment  IV,  (2) 
nature  and  complexity  of  the  task,  (3)  correspondence 
between  imagery  and  the  written  or  spoken  text,  (4) 
degree  of  explicitness  of  the  images  supplied,  (5) 
development  of  associations  between  images  and  words. 


1.  Authors:  Bickley,  William  R.  &  Bynum,  James  A. 


2.  Title:  Training  Device  Effectiveness:  Formulation  and 
Evaluation  of  a  Methodology. 

3.  Source:  U.S.  Army  Research  Institute  Research  Report, 
September  1980. 

4.  Topic  Keywords:  Training  Effectiveness  Ratio  ; 

Transfer  of  Training  ;  Helicopter  Flight  Simulation  . 

5.  Short  Summary:  A  formula  devised  for  measuring 
effectiveness  of  simulation  training  was  tested  on  a 
helicopter  flight  simulator  and  found  valid  on  a  per 
maneuver  basis. 

6.  Devices: 

a.  AH-1  helicopter. 

b.  AH1FS  high  fidelity  flight  simulator. 

7.  Institutions: 

a.  Sponsor:  U.  S.  Army. 

b.  Performing  Organization:  U.S.  Army  Research 
Institute  Field  Unit,  Fort  Rucker,  AL  36362. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  groups:  2. 

b.  Description  of  groups: 

(1)  Subjects:  Approximately  21  rated  Army 
helicopter  pilots  ("approximately,"  because  data 
from  some  individuals  on  particular  maneuvers  was 
discarded  since  training  on  those  maneuvers  fell 
outside  specifications) . 

(2)  Controls:  14  same. 

c.  Tests  or  trials/timing:  Post-tests. 

d.  Number  of  different  types  of  measures:  1. 

e.  Description  of  measurements  and  ratings:  Trials  to 
criterion  (criterion  measured  by  instructor  pilots  on  a 
9-point  scale,  with  9  signifying  criterion 
proficiency) . 

f.  Experimental  setting/training  context: 
institutional,  hands-on. 
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g.  Statistical  methods:  regression  analysis;  SPSS 
program  NONLINEAR  (Robinson  1977);  Marquardt's  method. 

h.  Variables  being  manipulated: 

(1)  Training  devices,  as  above,  section  6.  Chief 
independent  variable  was  number  of  device  training 
trials  prior  to  aircraft  training  to  criterion; 
for  each  maneuver,  there  were  three  different 
levels  of  training. 

(2)  Fidelity  levels  of  simulator: 

(a)  Physical:  high 

(b)  Functional:  high 

(3)  Type  of  task/skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  part-task. 

(4)  Task  difficulty:  high 

i.  Stage  of  training:  transition 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  not  specified;  assumed  intensive 

(2)  Features  used:  Freeze  capability; 
Restart/resequence  capability;  Malfunction 
selection;  Automated  demonstration; 

Record/ playback 

Abstract : 

a.  Study  Synopsis:  A  limitation  of  the  Cumulative 
Transfer  Effectiveness  Ratio  proposed  by  Roscoe  (1971) 
is  its  high  dependence  on  a  specific  amount  of 
simulator  training.  This  study  sought  to  validate  a 
model  which  would  calculate  simlator  training 
effectiveness  for  all  amounts  of  simulator  training,  by 
applying  it  to  the  case  of  a  simulator  of  heliopter 
flight. 

The  proposed  formula  is  expressed  as  y  =  ae  -  bx  +  c, 
where  y  is  the  amount  of  aircraft  training  required 
after  simulator  training,  a  is  an  arbitrary  constant,  b 
is  the  proportional  constant,  and  x  is  the  amount  of 
simulator  training  given.  It  was  applied  to 
experimental  results  of  another  study  (Provenmire  and 
Roscoe  1973)  and  a  good  rough  fit  of  the  formula  with 
the  evidence  was  obtained. 

In  the  present  experiment,  rated  Army  helicopter  pilots 


transitioning  to  the  AH-1  helicopter  were  divided  into 
two  groups,  of  which  one  received  training  only  in  the 
aircraft  (that  is,  simulated  training  =  0) ,  while  the 
other,  experimental  group,  received  training  in  the 
simulator  prior  to  aircraft  training.  Within  the 
simulator,  three  different  levels  of  training — in  terms 
of  number  of  trials--  were  given  for  each  of  31 
specific  maneuvers,  such  that  the  amount  of  training 
time  per  maneuver  could  be  compared  with  subsequently 
demonstrated  proficiency  in  the  aircraft,  for  that 
maneuver.  Proficiency  measurement  was  the  number  of 
aircraft  trials  required  to  reach  criterion  for  the 
maneuver . 

b.  Results:  Except  for  3  maneuvers,  25  to  80  percent 
of  the  variance  in  aircraft  training  amounts  required 
to  reach  criterion  could  be  accounted  for  by  the  amount 
of  prior  simulator  training  for  the  maneuver. 

The  "goodness  of  fit"  of  the  theoretical  model  for 
measuring  training  effectiveness  was  not  easy  to  judge, 
expecially  as  the  results  of  several  of  the  maneuvers 
were  very  inconclusive  due  to  obvious  overtraining  in 
the  simulator  which  led  to  asymptotic  performance  in 
the  aircraft.  However,  variance  from  performance  as 
described  by  the  model  was  less  than  significant  at  the 
a=.05  level  on  31  maneuvers. 

c.  Authors'  Conclusions:  It  should  be  pointed  out 
there  are  other  models  and  theoretical  functions  that 
would  fit  the  data  just  as  well  or  even  better. 

However,  there  is  no  cogent  reason  for  rejecting  the 
model  under  consideration  as  a  viable  heuristic.  From 
it,  cost  effectiveness  of  simulator  time  vis-a-vis  time 
in  the  aircraft  can  be  calculated. 

Regardless  of  the  intrinsic  characteristics  of  the 
simulator,  its  effectiveness  is  a  function  of  how  it  is 
used.  The  quantitative  measures  of  effectiveness 
determined  by  this  study  are  very  much  a  function  of 
how  the  instructor  used  the  simulator  as  a  training 
device.  As  simulator  instructional  tactics  are 
refined,  the  device  effectiveness  should  improve.  The 
tradeoff  curves  determined  by  the  study  represent  not 
the  optimum  effectiveness  of  this  device,  but  the 
baseline  effectiveness. 


1.  Author:  Biersner,  Robert  J. 

2.  Title:  Attitudes  and  Other  Factors  Related  to  Aviation 
Maintenance  Training  Effectiveness. 

3.  Source:  Defense  Technical  Information  Center  Technical 
Report  CNETS  6-75,  December  1975. 

4.  Topic  Keywords:  Maintenance  Training  ;  Attitudes  ; 
Reinforcement  ;  Individualized  Training  ;  Fidelity  ; 

Media  Comparison  . 

5.  Short  Summary:  No  significant  differences  were  found 
between  subjects  trained  on  a  moderate  fidelity  maintenance 
training  device  and  controls  trained  on  modified  operational 
equipment,  despite  the  fact  that  the  subjects  trained  on  the 
simulator  were  younger  and  less  experienced.  The  subjects 
were  tested  for  maintenance  parts  knowledge  and  some 
maintenance  procedures,  as  well  as  their  attitudes  toward 
the  training  devices. 

6.  'Devices: 

a.  EC2LP  Simulator  of  Maintenance  Equipment  -  (used  by 
experimental  subjects) . 

b.  MTU  Maintenance  Training  Unit  (Operational 
Equipment  Trainer  used  by  controls)  . 

7.  Institutions: 

a.  Sponsor: 

Naval  Education  and  Training  Support  Command, 

Pensacola,  FL. 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Controls:  41  Enlisted  male  trainees  attending 
O-level  maintenance  courses 

(2)  Subjects:  30  same 

c.  Measures  and  Timing  of  Measures:  Pre-Test,  Post- 
Test,  and  Post-Questionnaire 

d.  Number  of  Different  Types  of  Measurements  Used:  3 

e.  Description  of  Measurements  and  Ratings: 


(1)  Training  Device  Questionnaire 

(2)  Written  pre-test  and  post-test  (not  identical 
for  any  one  trainee) ;  multiple  choice 

(3)  Performance  Post-Test 

f.  Experimental  Setting/Training  Context: 

Institutional  classroom 

g.  Statistical  Methods:  T-tests;  Pearson 
product-moment  correlations;  centroid  solution  to 
varimax  rotation  of  items  (questionnaire) 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  EC2LP  Simulator  of 
Maintenance  Equipment;  MTU  Maintenance  Training 
Unit  (Operational  Equipment  Trainer) 

(2)  Fidelity  Levels: 

(a)  Physical:  Moderate-High  (of  simulator) 

(b)  Functional:  High  (of  simulator) 

(3)  Type  of  Task:  Cognitive  maintenance 

(4)  Task  Difficulty:  Medium 

(5)  Skills  Required  by  Task:  Perceptual, 
cognitive 

i.  State  of  Training:  Familiarization 

j.  Trainee  Sophisticitation :  Intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  Lockstep 

l.  User  Acceptance/Attitude:  Moderately  to  highly 
favorable 

m.  Use  of  Instructional  Features:  Very  incomplete 
Abstract : 

a.  Study  Synopsis:  This  study  measured  both  the 
proficiency  and  attitudes  toward  training  devices  of 
subjects  trained  on  one  of  two  different  devices  being 
used  for  teaching  maintenance  knowledge  factors.  The 
cont  rol  group,  using  operational  equipment  somewhat 
modified  for  training  purposes  (the  MTU  of  Maintenance 
Training  Unit)  was  in  general  more  experienced  and 
older  than  the  experimental  subjects,  who  used  a 
simulator  ( EC2LP)  of  moderately  high  physical  fidelity. 

Testing  consisted  of  pre-  and  post-  written  tests 
(multiple  choice,  in  which  two  similar  tests  were 
alternated  between  half  the  trainees  before  and  after, 
so  that  no  trainee  took  the  same  test  twice,  but  each 
test  was  taken  by  equal  numbers  both  before  and  after) 
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and  a  post-test  of  performance.  All  trainees  also 
filled  out  a  Training  Device  Questionnaire,  the  results 
of  which  were  analyzed  into  6  attitudinal  factors: 
Reinforcement,  Individualized  Training,  Media 
Comparison,  Training  Deficiencies,  Utility,  and 
Training  Fidelity. 

(1)  Performance  on  tests:  the  two  groups  showed 
no  significant  differences  related  to  the  training 
devices.  Some  test  scores,  however,  were  found  to 
be  significantly  related  to  the  General 
Classification  Test  and  the  Mechanical  Test  of  the 
Basic  Test  Battery;  also  to  time  in  squadron  (a 
measure  of  OJT)  and  to  age. 

(2)  Attitudes: 

(a)  Trainees  who  rated  the  training  devices 
better  than  books  and/or  charts  performed 
better  on  both  the  written  and  performance 
tests  than  those  trainees  who  judged  books 
and  charts  to  be  better  than  the  training 
devices . 

(b)  Trainees  who  were  older  and  who  had  been 
in  the  Navy  longer  rated  the  two  training 
devices  as  less  suitable  for  individualized 
training  than  did  younger  trainees  who  had 
been  in  the  service  for  a  shorter  period; 
trainees  who  had  higher  verbal  intelligence 
(GCT)  and  higher  mechanical  aptitude  (MECH) 
scores  rated  both  of  the  devices  as  less 
deficient  for  training  periods  than  those 
with  lower  scores;  and  trainees  with  higher 
arithmetic  and  mechanical  aptitudes  judged 
the  devices  higher  in  fidelity  than  did  those 
with  lower  scores  in  these  aptitudes. 

(c)  Both  trainees  and  instructors  favored  the 
simulator  over  the  operat : onal  equipment  for 
training  in  every  case. 

Highly  pertinent  to  the  issue  of  simulation  fidelity 
was  the  attitude  of  trainees  regarding  deficiencies  in 
the  operational  equipment  trainee  vs.  the  simulator. 
These  findings  indicate  that  "although  the  EC2LP 
simulators  have  low  structural  fidelity,  these 
simulators  possess  high  functional  fidelity  which 
probably  provides  a  better  description  than  the  MTUs  of 
the  integration  of  critical  equipment  parts  with  each 
other  and  with  the  aircraft  as  a  whole....  It  is 
likely...  that  differences  in  this  training  attitude 


were  based  largely  on  preconceived  differences  in  the 
functional  fidelity  of  the  two  devices." 
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teaching  the  night  landing  task  via  a 
tr ansf er-of- training  design,  duri.ig  -he  carrier 
qualification  phase  of  Navy  jet  p„j.ot  training.  The 
subject  group  of  26  trainees  received  device  training 
prior  to  FCLP  and  carrier  qualification  flights  in  the 
A7E  aircraft.  The  control  group  of  27  trainees 
received  no  device  training.  Within  each  group  there 
was  a  subgroup  of  relatively  inexperienced  pilots 
( "Nuggets ") ,  13  in  the  experimental  group  and  16  in  the 
control  group. 

Three  different  types  of  performance  measures  were 
taken  during  the  FCLP  and  Carrier  Qualification 
training  flights  in  the  A7E:  (1)  Attrition  rate  from 
the  carrier  qualification  program;  (2)  landing  ship 
officer  scoring  of  candidates;  (3)  objective  measures, 
of  which  the  most  important  were  the  Boarding  Rate 
(percentage  of  final  approaches  that  resulted  in 
successful  landings)  and  the  Landing  Performance  Score, 
an  index  of  pilot  performance  derived  from  wire 
arrestment  and  bolter  or  waveoff  data. 

b.  Results;  Summary  of  A7E  Landing  Performance 
Measures  and  Statistical  Results. 


(1)  Key  Data: 


Mean 

Scores 

Stat. 

Actual 

NCLT 

No-NCLT 

Sig . 

Diff . 

Objective  Measures 

Landing  Performance 
Day  CQ  4.6  0 

Score 

4.65 

N.S. 

-0.05 

Night  CQ  4.27 

3.90 

p  <  .  0  2 

+  0.37 

Boarding  Rate 

Day  CQ  91.4 

90.4 

N.S. 

+  1% 

Night  CQ  76.7 

69.3 

p<  .05 

+7% 

Attrition  Measures 

Attrition  First  CQ 

4% 

3  0% 

-26% 

Success  First  CQ 

96% 

70% 

p-  .006 

+  26% 

LSO  Measures 

Day  FCLP  2.88 

2.90 

N.S. 

-0.02 

Night  FCLP  2.88 

2.81 

N.S. 

+  0.07 

Day  CQ  2.84 

2.84 

N.S. 

0.0 

Night  CQ  2.76 

2.47 

p  ■  .003 

+  0.29 

(2)  Verbal  description:  As  indicated  in  the  table 
above,  the  performance  of  the  device- trained  group 
in  the  night  landing  task  during  carrier 
qualification  was  significantly  superior  to  the 
non-device- trained  group  in  some  of  the  principal 
performance  measures;  of  which  the  attrition  rate 
had  probably  the  greatest  practical  importance. 
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e.  Description  of  measurements  and  ratings: 

(1)  Attrition  rate 

(2)  Subjective: 

(a)  Landing  Ship  Officer  scores 

(b)  Student  questionnaires 

(3)  Objective: 

(a)  Approach  performance  score 

(b)  Landing  performance  score 

(c)  Boarding  rate 

(d)  Bolter  rate 

(e)  Wire  arrestment 

f.  Experimental  setting/training  context: 
institutional,  hands-on. 

g.  Statistical  methods:  t-test;  others  not  specified 


Variables  being  manipulated 

(1)  Training  devices:  as  in  section  6  above 
Subjects  received  training  time  on  device  prior 
FCLP  training;  controls  received  no  device 
training,  nor  substitute  aircraft  training 

(2)  Fidelity  levels 

(a)  Physical:  high 

(b)  Functional:  medium  high  to  high 

(3)  Type  of  task/skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  procedural, 
part- task 

(4)  Task  difficulty:  medium-high  to  high 


Stage  of  training:  skill,  transition,  advanced 
Trainee  sophistication:  intermediate 
Incorporation  of  device  into  P.O.I.:  lock-step 

User  acceptance  or  attitude 

(1)  Instructor:  favorable 

(2)  Students:  highly  favorable  as  measured  by 
questionai re 

Use  of  instructional  features 

(1)  Intensity:  intensive  (assumed,  not  specified) 

(2)  Features  used:  Freeze  capability; 
Number/quality  of  reponses;  Cue  enhancement; 
Restart/resequence  capability,  modified  to  reenter 
only  at  one  point 


Abstract : 


a.  Study  Synopsis:  This  study  evaluated  the 
effectiveness  of  a  Night  Carrier  Landing  Trainer 
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1.  Authors:  Brictson,  Clyde  A.  &  Burger,  William  J. 


2.  Title:  Transfer  of  Training  Effectiveness:  A7E  Night 
Carrier  Landing  Trainer  (NCLT)  Device  2F103. 

3.  Source:  NAVTRAEQUIPCEN  Technical  Report  74-C-0079-1, 
August  1976. 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Training  Effectiveness  ;  Flight  Simulation  ; 

Night  Carrier  Landing  . 

5.  Short  Summary:  A  transf er-of-tr aining  evaluation  of  a 
Night  Carrier  Landing  Trainer  found  the  simulator  effective 
in  teaching  the  night  carrier  landing  task  to  student  pilots 
in  the  carrier  qualification  phase  of  training. 

6.  Devices: 

a.  A7E  jet  (transfer  aircraft) 

b.  NCLT  Night  Carrier  Landing  Trainer,  a  part-task 
trainer  with  simulated  cockpit,  visual  display  system, 

3  D.O.F.  motion  system,  instructor  console,  and 
digital  computer. 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Code 
N-215,  Orlando,  FL  32813. 

b.  Performing  Organization:  Dunlap  and  Associates, 
Inc.,  Western  Division,  La  Jolla,  CA  92037. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  groups:  2 

b.  Description  of  groups: 

(1)  Subjects:  26  trainees  in  advanced  navy  jet 
pilot  training  (carrier  qualification) 

(a)  13  relatively  inexperienced  pilots 

(b)  13  more  experienced  pilots 

(2)  Controls:  27  same 

(a)  16  relatively  inexperienced 

(b)  11  more  experienced 

c.  Tests  or  trials/timing:  Post-tests  (FCLP  training 
and  carrier  qualification  training  flights) 

a.  Number  of  different  types  of  measures:  3 


b.  Results:  Both  the  mean  and  the  standard  deviation 
of  the  lateral  position  were  significantly  larger 

(p - .01)  in  the  simulator  than  in  the  car,  for  both 
experienced  and  inexperienced  drivers.  The  standard 
deviations  of  the  yaw  rate  and  steering  wheel  angle 
were  significantly  smaller  (p<.01)  in  the  simulator 
than  in  the  car  for  the  experienced  drivers,  whereas  no 
significant  differences  were  found  between  these 
variables  for  the  inexperienced  drivers.  Greater 
driving  experience  did  not  affect  the  mean  lateral 
position,  but  resulted  in  significantly  smaller  (p<.01) 
standard  deviations  for  the  lateral  position,  yaw  rate, 
and  steering  wheel  angle  in  the  simulator.  Driving 
experience  had  no  effect  on  those  measures  in  the 
instrumented  car.  The  inexperienced  drivers  had  a 
higher  spectral  density  than  the  experienced  drivers. 
This  indicates  a  greater  expenditure  of  energy  in 
steering.  Greater  experience  produces  significantly 
smaller  standard  deviations  (p<.05)  for  velocity  and 
accelerator  position  in  the  simulator  and  the 
instrumented  car.  The  experienced  drivers  maintained  a 
more  constant  velocity. 

Questionnaire  opinions  discriminated  consistently  and 
significantly  between  the  simulator  and  the 
instrumented  car  for  both  groups.  Drivers  judged  the 
simulator  considerably  more  unfavorable  than  the 
instrumended  car,  with  an  exception  for  control  of 
speed.  Experienced  drivers  gave  more  favorable 
judgements  than  the  inexperienced  drivers  (p'.Ol)  on 
speed  control. 

There  was  a  significantly  high  behavioral  correlation 
between  the  two  systems  for  all  drivers  (Pearson 
product-moment  correlations  were  .36  for  the  mean 
lateral  position,  .57  for  the  standard  deviation  of  the 
lateral  position,  .14  for  the  standard  deviation  of  the 
yaw  rate,  and  .32  for  the  standard  deviation  of  the 
steering  wheel  angle,  all  significant,  p<.05,  except 
for  the  standard  deviation  of  the  yaw  rate) . 

c.  Author's  Conclusions:  A  fixed-base  simulator 
offers  a  valid  method  for  studying  straight  road 
driving.  Also,  the  simulator  is  more  sensitive  to 
differences  between  levels  of  driving  experience  than 
is  the  instrumented  car  on  the  road. 
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g.  Statistical  methods:  Pearson  product-moment 
correlations;  Newman-Keuls  tests 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  in  section  6  above. 

Other  variables: 

(a)  Lateral  control  requirement  (suggested  to 
one  group  within  each  main  group  by 
experimenter ) 

(b)  "Longitudinal"  control  requirement  (i.e., 
speed) 

i.  Stage  of  training:  introduction 

j.  Trainee  sophistication:  novice  and  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  not 
applicable 

l.  User  acceptance  or  attitude: 

(1)  Instructors:  not  applicable 

(2)  Subjects:  moderately  favorable  to  unfavorable 

m.  Use  of  instructional  features 

(1)  Intensity:  intensive 

(2)  Features  used:  Number/quality  of  responses; 
others  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  This  study  was  designed  to 
evaluate  the  validity  of  an  automobile  driving 
simulator  in  the  way  in  which  it  reproduced  a 
behavioral  environment. 

A  group  of  24  very  inexperienced  drivers  was  compared 
with  a  group  of  experienced  drivers  in  performance  in 
driving  along  a  straight  road  both  in  the  simulator  and 
in  an  instrumented  car.  Measurements  were  taken  of  the 
following  dependent  variables  reflecting  vehicle 
control:  steering  wheel  angle,  lateral  position 

(distance  between  driver  and  right  lane  marker),  yaw 
rate,  position  of  acclerator,  and  velocity. 

All  subjects  were  also  given  questionnaires  for 
evaluation  of  the  device  as  to  realism,  difficulty, 
required  attention,  and  monotony. 

Task  demands,  in  terms  of  verbal  requests  from  the 
experimenter  for  straight  or  constant-speed  driving,  or 
both,  were  other  independent  variables  of  the  study. 
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1.  Author:  Blaauw,  Gerald  J. 

2.  Title:  Driving  Experience  and  Task  Demands  in  Simulator 
and  Instrumented  Car:  A  Validation  Study. 

3.  Source:  Human  Factors  ,  1982,  24(4),  473-486. 

4.  Topic  Keywords:  Driving  Simulation  . 

5.  Short  Summary:  A  driving  simulator  was  compared  with  an 
instrumented  car  with  drivers  of  varying  expertise.  The 
experiment  found  reasonable  validity  for  the  simulator  and 
determined  it  to  be  more  sensitive  than  the  car  to  driver 
experience. 

6.  Devices: 

a.  Fixed-base  Automobile  Driving  Simulator. 

b.  Instrumented  Car. 

7.  Institutions: 

a.  Sponsor:  Institute  for  Perception  TNO,  Kampweg  5, 
Postbus  23,  Soesterberg  3769  ZG,  The  Netherlands 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment. 

a.  Number  of  groups:  2 

b.  Description  of  groups: 

(1)  Subjects:  24  experienced  male  drivers 
(licensed  for  a  minimum  of  3  years,  had  minimum 
30,000  km  experience) 

(2)  Subjects:  24  inexperienced  drivers  (had  at 
most  just  passed  their  driving  test) 

c.  Tests  or  trials/timing:  2  in-process  trials  (one 
each  vehicle) 

d.  Number  of  different  types  of  measures:  2 

e.  Description  of  measurements  and  ratings 

(1)  Position  of  controls  (steering  wheel  and 
accelerator ) 

(2)  Movement  of  car  (lateral,  longitudinal,  yaw) 

f.  Experimental  setting/training  context:  laboratory, 
hands  on 
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than  in  a  comparable  airplane  under  roughly  comparable 
conditions.  We  cannot  say  with  certainty  why  this  is 
so.  The  data  lead  us  to  hypothesize  that  the  observed 
differences  between  actual  and  simulated  flight  are  due 
to  higher  levels  of  arousal  in  flight,  which  may  have 
antagonized  in  part  the  effects  of  a  depressant  drug. 

It  is  clear  from  data  in  this  and  other  experiments 
(others  cited  include  hypoxia  and  alcohol  stress) ,  that 
the  GAT-1  simulator  is  a  useful,  sensitive,  and 
comparatively  inexpensive  device  for  studies  of  the 
effects  of  mild  stress  on  pilot  performance.  These 
data  also  suggest,  however,  that  extrapolations  from 
simulated  to  actual  flight  must  be  made  with 
considerable  caution.  For  example,  it  appears  that 
experienced  pilots  in  this  study  were  able  to 
compensate  in  part  for  the  effects  of  a  depressant  drug 
when  they  were  confronted  with  the  threat  factors 
inherent  in  the  real  world. 


Data  on  lateral  and  angular  deviations  from  localizer 
and  glidepath  centerlines,  and  on  indicated  airspeed, 
were  recorded  by  an  instrument.  Data  were  evaluated  by 
analysis  of  variance;  correlations  among  the  measures 
were  evaluated  by  linear  correlation  analysis. 

b.  Results: 


(1) 

Key  Data: 

Dep. 

Signif icance 

Dose  Effect 

(in  mg 

Var . 

in  Aircraft 

in 

Aircraft 

Eest 

Intermed. 

Worst 

DLA 

0  .01 

100 

0 

200 

SLA 

0  .01 

0 

100 

200 

DGA 

n.  s. 

0 

200 

100 

SGA 

0.05 

0 

100 

200 

SDA 

n .  s . 

200 

100 

0 

SSA 

0.05 

100 

0 

200 

Dep. 

Signif icance 

Dose  Effect 

( in  mg 

Var . 

in  Simulator 

in 

Simulator 

Best 

Intermed. 

Worst 

DLA 

0  .05 

0 

200 

100 

SLA 

0  .05 

0 

100 

200 

DGA 

0.01 

0 

100 

200 

SGA 

0  .01 

0 

100 

200 

SDA 

0.05 

0 

100 

200 

SSA 

0.01 

0 

100 

200 

DLA 

=  mean  absolute 

deviation 

from  localizer 

centerline ; 

SLA 

=  RMS  variablity 

in 

localizer 

track 

DGA 

=  mean  absolute 

deviation 

from  glidepath 

centerline ; 

SGA 

=  RMS  variablity 

in 

gl idepath 

track 

DSA 

=  mean  absolute 

deviation 

from  command  airspeed; 

SSA  =  RMS  variability  in  airspeed. 


(2)  Verbal  Description:  With  regard  to  the 
effects  of  barbiturates,  it  was  immediately 
evident  that  the  drugs  exerted  significant 
decremental  effects  on  performance.  The  primary 
effect  of  drug  was  present  to  a  significant  degree 
in  all  six  dependent  variables  in  the  simulator 
data,  but  in  only  four  of  the  six  variables  in  the 
aircraft  data.  The  consistency  of  the 
drug-related  effects  was  higher  in  the  simulator 
data  as  well  (see  table  above). 

c.  Authors'  Conclusions:  The  effects  of 
pharmacological  stressors,  particularly  at  low  doses, 
may  be  observed  more  readily  in  the  GAT-1  simulator 


f.  Experimental  Setting/Training  Context: 

Experimental,  laboratory  and  field 

g.  Statistical  Methods:  ANOVA;  linear  correlation 
analysis 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  in  section  6  above; 
however,  the  variable  of  interest  was  the  drug 
dosage : 

(a)  0  dosage  placebo 

(b)  100  mg  sodium  secobarbitol 

(c)  200  mg  sodium  secobarbitol 

(2)  Fidelity  levels: 

(a)  Physical:  of  simulator,  unspecified, 
presumably  medium-high 

(b)  Functional:  of  simulator,  unspecified, 
presumably  high 

(3)  Type  of  Task/Skill  required:  aircraft 
piloting  operations;  cognitive,  psychomotor, 
perceptual,  procedural,  part-task  (aircraft 
landing  approach) 

(4)  Task  difficulty:  high 

i.  Stage  of  Training:  familiarization;  skill 

j.  Trainee  Sophistication:  expert 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  unspecified 
9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
assess  the  similarity  between  a  simulator  and  an 
airplane  in  measuring  the  effects  of  stress — in  this 
case,  varying  dosages  of  a  depressant  drug — on  pilot 
performance . 

Five  highly  experienced  professional  pilots,  given 
three  separate  dosages  of  sodium  secobarbitol  (0  mg 
placebo,  100  mg,  200  mg)  in  a  single-blind  design,  were 
required  to  make  instrument  landing  approaches  in  the 
Link-Singer  GAT-1  light  airplane  simulator,  and,  at  a 
different  site  and  time,  in  an  instrumented  Cessna 
model  172.  The  pilots  had  been  given  prior 
familiarization  with  both  simulator  and  airplane. 
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1.  Authors:  Billings,  Charles  E.,  Gerke,  Ralph  J.,  & 
Wick,  Robert  L.  Jr. 


2.  Title:  Comparisons  of  Pilot  Performance  in  Simulated 
and  Actual  Flight. 

3.  Source:  Aviation,  Space,  and  Environmental  Medicine, 

March  1975,  304-308. 

4.  Topic  Keywords:  Drug-related  Effects  ; 

Flight  Simulation  ;  Instrument  Landing  ; 

Instrument  Flight  Simulation  . 

5.  Short  Summary:  This  experiment  comparing  the 
performance  of  pilots  under  drug  stress  in  a  GAT-1  simulator 
and  an  airplane  suggests  that  the  GAT-1  simulator  may  be  a 
sensitive  device  for  studying  the  effects  of  mild  stress  on 
pilot  performance. 

6.  Devices: 

a.  Link-Singer  GAT-1  light  airplane  instrument  flight 
simulator 

b.  Cessna  Model  172  fully  equipped  for  instrument 
flight 

7.  Institutions: 

a.  Sponsor:  Aerospace  Medical  Research  Laboratory, 

Aerospace  Medical  Division,  Air  Force  Systems  Command, 

Wright  Patterson  AFB,  OH. 

b.  Performing  Organization:  Department  of  Preventive 
Medicine,  Ohio  State  University,  Columbus,  OH  43210, 

Man-Machine  Integration  Branch,  Ames  Research  Center, 

Moffet  Field,  CA  94035 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  5  highly  experienced 
professional  pilots 

c.  Tests  or  Trials/Timing:  2  test  flights 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings: 

Instrument-recorded  quantified  data:  airspeed  and 
deviations  from  landing  approach  path. 

_ _ _ _ -jh 
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were  being  taught  as  well  as  familiarization  with 
equipment,  although  the  former  was  not  officially 
recognized  as  an  objective  of  training. 
Cost-effective  use  of  the  simulator  requires 
matching  specific  capabilities  of  the  simulator 
with  identified  training  requirements.  When  the 
simulator  capabilities  are  not  incorporated  into 
the  design  of  the  training  program,  or  the 
training  requirement  are  not  precisely  identified, 
use  of  the  simulator  will  fall  short  of  optimum 
effectiveness. 

b.  Results:  No  statistical  measures  are  reported.  No 
salient  differences  in  effectiveness  were  observed 
between  the  simulator  and  conventional  classroom 
training,  according  to  the  author.  Subjectively,  both 
instructors  and  students  were  pleased  with  simulator 
use,  but  did  not  put  forward  arguments  clearly  favoring 
it  over  other  training  aids. 

c.  Authors'  Conclusions:  A  formal,  summative 
evaluation  of  the  training  effectiveness  of  the  ECII-LP 
simulators  should  probably  not  be  undertaken  until 
minor  modifications  have  been  made  in  the  design  and 
support  of  the  ECII-LPs,  and  some  major  modifications 
have  been  instituted  in  the  management  and  delivery  of 
the  existing  training  courses  for  pilots  and  Naval 
Flight  Officers.  At  the  time  of  the  study,  certain 
deficiencies  in  the  simulator  lowered  instructor  and 
student  morale,  and  instructors  were  not  sufficiently 
knowledgeable  about  the  simulator's  capabilities. 

These  problems  require  correction  at  a  technical  and  at 
a  formal  level. 


g.  Statistical  Methods:  none  used 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Flight  Cockpit  Display 
Simulator  ECII-LP 

(2)  Fidelity  Levels:  medium  to  high 

(3)  Type  of  Task:  familiarization  with  display 
and  some  procedural  tasks 

(4)  Task  Difficulty:  moderate 

i.  Stage  of  Training:  pre-flight 

j.  Trainee  Sophistication:  highly  varied 

k.  Incorporation  of  Device  into  P.O.I.: 
instructor-managed 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  instructors  complained  about 
technical  deficiencies  of  the  device 

(2)  Students:  good 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  incomplete 

(2)  Features  used:  interactive  capability; 
others  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  The  author  states,  "this  report  is 
essentially  a  formative  review,  based  on  observations 
and  interviews  rather  than  test  scores  or  other 
objective  quantifiable  information."  The  tenor  of  the 
observations  is  as  follows: 

(1)  Instructor  Findings:  The  simulator  made 
presentation  of  information  better  organized  and 
more  convenient  than  other  training  aids. 
Instructors  were  disturbed  by  a  technical 
deficiency  of  the  simulator  which  made  them  appear 
incompetent.  Instructor  Managed  Instruction  is 
indicated  in  preference  to  standard  lecture 
procedures. 

(2)  Student  Findings:  Students  receiving 
simulator  ining  were  less  confused  than  the 
controls,  as  indicated  by  fewer  and  more 
sophisticated  questions  and  other  subjective 
evidence . 

(3)  Other  Observations:  The  capabilities  of  the 
simulator  were  not  being  fully  realized;  in 
particular  the  computer-programmed  interactive 
capability.  The  skills  being  learned  were  not 
clearly  defined;  e.g.,  basic  cockpit  procedures 
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1.  Author:  Biersner,  Robert  J. 

2.  Title:  Observations  on  the  Use  and  Evaluation  of 
ECII-LP  Simulators  for  Aviation  Training. 

3.  Source:  Chief  of  Naval  Education  and  Training  Support 
Report  2-76,  1976. 

4.  Topic  Keywords:  Display  Systems  ;  Visual  Aids  ; 

Flight  Simulators  ;  Human  Factors  ;  Participant-Observer  . 

5.  Short  Summary:  Simulator  training  proved  to  be  no  more 
effective  than  conventional  classroom  training  for  learning 
the  subject  task.  However,  the  program  using  the  simulator 
was  hampered  by  lack  of  integration  into  the  program, 
instructors  lack  of  knowledge  of  the  training  device,  and 
other  correctible  shortcomings.  A  more  extended 
implementation  is  recommended. 

6.  Devices:  Flight  Cockpit  Simulator  ECII-LP 

7.  Institutions: 

a.  Sponsor:  Chief  of  Naval  Education  &  Training 
Support,  Pensacola,  FL. 

b.  Performing  Organization: 

NAVTRADET  1048,  Sherman  Field,  Naval  Air  Station, 
Pensacola,  FL. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  6  Navy  and  Marine  undergraduate 
student  pilots 

(2)  Controls:  30  Navy  and  Marine  undegraduate 
student  pilots 

c.  Tests  or  Trials/Timing:  post-test  evaluations 

d.  Number  of  Different  Types  of  Measures:  3 
( nonquanti tative  evaluations) 

e.  Description  of  Measurements  and  Ratings: 

(1)  Student  interviews 

(2)  Student  questionnaires 

(3)  Instructor  evaluation  reports 

f.  Experimental  Setting/Training  Context:  school 
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b.  Results: 


SIM  GROUP 

MTU  GROUP 

(N=30) 

(N=41 ) 

Standard 

Standard 

Mean  Deviation 

Mean 

Deviation 

Written  Pre-Test 

17.48  7.02 

18.33 

5.23 

Written  Post-Test 

34.35  3.95 

34.73 

3.75 

Performance  Test 

75.81  15.30 

77.75 

21.31 

Verbal  Description 

i:  "Training  effectivene 

S  S •  •  •  W3S 

similar  for  the  EC2LP  simulators  and  MTUs  despite 
differences  between  the  two  training  groups  in  age,  pay 
grade,  and  years  of  naval  service.  Previous  squadron 
experience,  which  is  an  index  of  on-the-job  training, 
was  found  to  improve  scores  on...  tests  for  both 
training  device  groups.  More  general  training 
experience...  appeared  to  be  especially  important  to 
better  performance  test  scores.  Trainee  acceptance  of 
the  two  devices  appears  to  be  related  not  only  to 
characteristics  of  the  devices,  but  to  intelligence, 
aptitudes,  and  previous  military  experience.... 

Trainees  with  higher  mechanical  and  arithmetic 
aptitudes  find  training  devices  to  be  high  in  training 
fidelity. " 

c.  Author's  Conclusion:  Apparently  the  two  training 
devices  compared  in  this  evaluation  can  be  used  in 
aviation  maintenance  courses  with  equal  training 
effectiveness.  The  substitution  of  the  EC2LP  simulator 
for  the  MTU  would  be  a  highly  cost-effective  procedure. 
This  recommendation,  however,  assumes  that  the  criteria 
of  training  effectiveness  (written  and  performance 
tests)  are  valid  measures  of  the  training  objectives. 

The  absence  of  structural  fidelity  in  the  simulators 
does  not  impair  trainee  attitudes  toward  the  device. 
Instructor  attitudes,  as  well  as  design  features  of  the 
simulator,  indicate  that  this  device  could  be  used  more 
for  individualized  training. 
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Other  measures  not  shown  in  the  table  above  showed 
either  no  significant  differences  between  groups, 
or  showed  superiority  for  the  experimental, 
device-trained  group. 

An  especially  interesting  finding  was  that  the 
device  training  had  no  measurable  significant 
effect  on  performance  in  day  landings. 

The  difference  between  the  relatively  untrained 
"Nugget"  subgroups  was  even  more  pronounced  than 
for  the  main  groups;  almost  half  of  the  Nuggets 
(44%)  in  the  no-NCLT  group  failed  to  qualify  on 
the  carrier,  compared  with  only  8%  of  the 
device-trained  Nuggets;  and  the  one  failure  among 
the  device- trained  Nuggets  was  disqualified  on  the 
day  portion. 

c.  Authors'  Conclusions:  It  is  clear  that  positive 
NCLT  transfer  of  training  was  demonstrated.  This  was 
especially  true  for  the  less  experienced  pilots.  The 
data  suggest  that  NCLT  training  can  be  combined  with 
FCLP  training  and  in  some  cases  perhaps  be  used  to 
supplant  some  FCLP  night  work.  This  would  appear  more 
feasible  with  the  more  experienced  pilots  as  these  were 
scored  consistently  higher  than  the  less  experienced 
pilots  by  the  Landing  Ship  Officers  across  all  FCLP 
periods. 

Day  FCLP  and  CQ  performance  did  not  change  as  a  result 
of  NCLT  training.  This  seems  to  indicate  that  NCLT 
training  in  night  procedures  and  visual  cues  does  not 
transfer  to  day  landing  performance. 
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1.  Authors:  Browning,  Robert  F. ,  Ryan,  Leonard  E. , 

Scott,  Paul  G. ,  &  Smode,  Alfred  F. 

2.  Title:  Training  Effectiveness  Evaluation  of  Device 
2F87F,  P-3C  Operational  Flight  Trainer. 

3.  Source:  Training  Analysis  and  Evaluation  Group  Report 
No.  42,  January  1977. 

4.  Topic  Keywords:  Visual  Flight  Simulation  ; 

Transfer  of  Training  ;  Proficiency-based  Training  ; 

Training  Effectiveness  . 

5.  Short  Summary:  This  transf er-of-training  comparison  of 
a  high  fidelity  flight  simulator  with  an  older,  more  limited 
flight  simulator  found  that  the  high  fidelity  simulator  was 
more  effective  as  a  trainer. 

6.  Devices: 

a.  P-3  turboprop  aircraft  -  the  transfer  environment 

b.  Device  2F87F  ,  High  Fidelity  Digital  Operational 
Flight  Trainer,  with  visual  and  6  degr ees-of-f reedom 
platform  motion  capability. 

c.  Device  2F69D  Analog  Operational  Flight  Trainer  with 
no  visual  simulation  and  3  degrees-of-f reedom  motion. 

7.  Institutions: 

a.  Sponsor:  Training  Analysis  and  Evaluation  Group, 
U.S.  Navy,  Orlando,  FL  32813. 

b.  Performing  organization:  same 

8.  Type  of  Article:  Experiment. 

a.  Number  of  groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  27  newly  designated  first-tour 
naval  aviators 

(2)  Controls:  16  same  concurrent  controls 

(3)  58  same  historical  controls 

c.  Tests  or  trials/timing:  post-tests 

d.  Number  of  different  types  of  measures:  4 

e.  Description  of  measurements  and  ratings 

(1)  Check  flight  grades 
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(2)  Number  of  flights  and  landings  to  criterion 
prof iciency 

(3)  Number  of  specific  tasks  on  which  students 
were  judged  proficient 

(4)  Errors  per  landing 

f.  Experimental  setting/training  context: 

Institutional,  hands-on 

g.  Statistical  methods:  not  specified 

h.  Variables  being  manipulated 

(1)  Training  devices:  as  above  section  6 

(2)  Fidelity  levels 

Physical  Functional 

2F87F  High  High 

2F69D  Medium  Medium 

(3)  Type  of  task/skill  required:  aircraft 
piloting  operations,  cognitive,  psychomotor, 
perceptual,  whole-task 

(4)  Task  difficulty:  High 

r 

i.  Stage  of  training:  transition 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude: 

(1)  Instructors:  very  favorable  as  measured  by 
questionnaire 

(2)  Students:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  incomplete 

(2)  Features  used:  not  specified 

9 .  Abstract: 

a.  Study  Synopsis:  This  study  undertook  to  compare 
the  training  effectiveness  of  a  new  state-of-the-art 
high  fidelity  simulator,  Device  2F87F,  with  that  of  an 
older,  less  versatile  and  realistic  simulator,  Device 
2F69D,  in  transitioning  first-tour  naval  aviators  to 
the  P-3  turboprop  aircraft.  The  chief  difference 
between  the  two  devices  was  the  visual  flight 
capability  of  the  newer  2F87F,  whereas  the  2F69D  had  no 
visual  capability. 

A  group  of  27  trainees  were  trained  on  the  2F87F,  while 
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concurrently  a  group  of  16  control  subjects  were 
trained  on  the  older  2F69D;  an  "historical"  group  of 
58  control  subjects,  previously  trained  on  the  same 
syllabus,  was  also  compared  with  the  experimental 
subjects  on  certain  measures.  Classroom  and  cockpit 
familiarization  and  procedures  instruction  was  the  same 
for  both  groups.  However,  the  experimental  group 
received  six  sessions  in  the  2F87F  simulator,  in 
accordance  with  a  new  syllabus  designed  for  that 
device,  whereas  the  control  group  received  only  3 
sessions  in  the  2F69D  simulator. 

The  performance  of  both  groups  were  measured  after 
transfer  to  P-3  aircraft  flying  training,  in  terms  of 
checkflight  grades,  number  of  aircraft  flights  and 
landings  to  criterion  proficiency,  number  of  check 
tasks  in  which  students  were  judged  proficient,  and 
errors  during  landings. 

b.  Results: 

(1)  Key  Data:  Flight  Hours  and  Flight  Grades  of 

Control  and  Experimental  Groups 

Historical  Concurrent  Experimental 


Controls 

Controls 

Sub j  ects 

Number  of 
Students 

58 

16 

27 

UPT  Basic  & 
Advanced 

55.8 

55.8 

54.2 

Flight 

Flight  Hrs. 
per  Student 
(P-3) 

15.1 

14.5 

8.8 

Check  Flight 
Grade  (P-3) 

3.02 

3.02 

3  .03 

**  averages  ** 


(2)  Verbal  Description:  The  experimental  group 
average  of  8.6  hours  in  aircraft  represents  a 
savings  of  40.6  hours  over  the  concurrent  controls 
and  43  percent  over  the  historical  controls.  The 
average  check  flight  score  did  not  differ 
significantly  from  that  of  the  control  groups. 

In  regard  to  proficiency  on  specific  check  tasks 
during  aircraft  flight,  every  one  of  20  check 
tasks  was  judged  proficient  for  the  experimental 
group  in  fewer  flights  than  for  the  concurrent 
control  group  (no  data  from  the  historical  control 
group  was  available  on  this  measure)  .  A 
comparison  of  the  Standard  Deviations  of  the 


experimental  and  control  group  suggests  that 
Device  287F  reduces  the  average  training  time 
difference  between  "fast"  and  "slow"  learners. 

The  average  number  of  aircraft  landings  for  the 
experimental  group  was  36;  for  the  Control  group, 
52;  a  31%  savings. 

A  cost  analysis  was  also  conducted  which  showed 
the  savings  in  aircraft  flying  time  resulting  from 
the  use  of  the  new  simulator  would  yield 
substantial  savings,  far  offsetting  the  increased 
cost  of  the  new  simulator. 

In  only  50  instances  out  of  1,200  gradings  were 
students  given  a  below  average  grade  on  a  task 
that  had  previously  been  graded  proficient.  The 
subsequent  lowering  of  a  grade  to  below  average 
after  proficiency  is  achieved  occurred  less  than 
5%  of  the  time. 

c.  Author's  Conclusions:  The  combination  of  six 
simulator  and  four  aircraft  flights  will  maintain 
current  standards  and  achieve  a  $40  million  savings 
over  a  10-year  period. 

Flight  time  can  be  reduced  by  training  each  task  to 
proficiency  in  Device  2F87F  prior  to  that  task  being 
checked  or  trained  in  the  aircraft.  The  ultimate 
reduction  in  aircraft  training  time  may  be  achieved 
only  through  proficiency-based  training.  Although 
conversion  to  a  train-to-pr of iciency  concept  presents  a 
number  of  formidable  problems,  it  has  merit  and 
warrants  strong  consideration. 


1.  Authors:  Burger,  William  J.  &  Brictson,  Clyde  A. 

2.  Title:  A7E  Transfer  of  Training  Effectiveness:  Device 
2C15A  CPT  and  Device  2F84B  OFT/WST. 

3.  Source:  NAVTRAEQUIPCEN  74-C-0079-2,  August  1976 

4.  Topic  Keywords:'  Transfer  of  Training  ; 

Flight  Simulation  ;  Training  Effectiveness  . 

5.  Short  Summary:  A  qualitative  evaluation  of  two  flight 
simulators  reveals  limitations  of  the  devices  and  recommends 
some  steps  for  improvement;  however,  the  qualitative 
methodology  fails  to  determine  the  training  efficiency  of 
the  devices. 

6.  Devices: 

a.  2C15A  Cockpit  Procedures  Trainer 

b.  2F84B  Operational  Flight  Trainer  /  Weapon  System 
Trainer 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization:  Dunlap  and  Associates, 
Inc.,  Western  Div.,  115  So.  Oak  St.,  Inglewood,  CA 
90301 

8.  Type  of  Article:  Survey. 

Researchers  reviewed  training  methods,  materials,  and 
observed  actual  training  sessions.  Interviews  were 
conducted  with  experienced  Phase  Officers  and  13  Instructor 
Pilots . 

a  -  d:  not  applicable 

e.  Description  of  Measurements  and  Ratings: 
subjective  assessments  of  device  effectiveness  and 
characteristics 

f.  Experimental  Setting/Training  Context:  not 
applicable 

g.  Statistical  Methods:  not  applicable 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 


(2)  Fidelity  Levels: 

CPT  OFT/WST  (a)  Physical:  High  High  (b) 
Functional:  Low  Medium  (3)  Type  of 

Task/Skill  Required: 

(a)  CPT  -  perceptual,  cognitive,  procedural 

(b)  OFT/WST  -  operations,  cognitive, 
psychomotor,  procedural 

04)  Task  Difficulty: 

i.  Stage  of  Training: 

(1)  CPT  -  introduction,  nomenclature,  procedural, 
familiarization 

(2)  OFT/WST  -  skill,  transition 

j.  Trainee  Sophistication:  novice  for  CPT; 
intermediate  for  OFT/WST 

k.  Incorporation  of  Device  into  P.O.I.:  not 
specified,  assumed  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  generally  not  favorable  to  CPT; 
moderately  favorable  to  OFT/WST 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  incomplete  due  to  equipment 
malfunctions  and  instructor  distrust 

(2)  Features  used: 

CPT:  not  specified 

OFT/WST:  Freeze  capability;  Record/Playback 

9.  Abstract: 

a.  Study  Synopsis:  The  study  objective,  to  assess  the 
transfer  of  training  effectiveness  of  two  A7E  aircraft 
simulators,  was  hindered  by  barriers  to  conducting  a 
controlled  quantitative  study.  The  researchers  chose 
as  an  option  a  qualitative  methodology  developed  by 
G.G.  Jeantheau,  described  as  "Level  One  Evaluation, 
Qualitative  Assessment,"  comprising  the  following 
parts : 

(1)  establishing  the  purpose  of  training; 

(2)  identifying  design  features  that  contribute  to 
or  hinder  training  effectiveness; 

(3)  identifying  features  of  training  practice  of 
use  of  the  device  which  contribute  to  or  hinder 
training . 

The  researchers  reviewed  training  methods  and 
materials,  observed  actual  training  sessions,  and 
conducted  intensive  interviews  with  Phase  Officers  and 
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experienced  Instructor  Pilots.  The  study  focussed  on 
specific  design  and  utilization  factors  rather  than  on 
overall  effectiveness,  since  the  subjective  nature  of 
the  evaluation  dictated  that  attempts  at  assessing  the 
latter  would  produce  inconclusive  results  at  best. 

b.  Results:  One  important  finding  of  the  study  was 
the  consensus  among  all  personnel  involved  that  the 
Cockpit  Procedures  Trainer  was  overdesigned  and  overly 
expensive  for  the  simple  nomenclature  and  procedural 
tasks  which  it  taught.  The  Operational  Flight 
Trainer/Weapons  Systems  Trainer  was  judged  by 
instructors  to  be  indispensable  for  pilot  training,  yet 
a  large  number  of  deficiencies  were  cited  that  begged 
for  improvement. 

c.  Authors'  Conclusions: 

(1)  Devices:  The  current  Cockpit  Procedures 
Trainer  is  best  left  alone.  Its  contribution  to 
overall  training  is  relatively  small,  and 
enhancement  of  its  capability  appears  more  costly, 
even  if  feasible,  than  the  potential  payoffs  seem 
to  warrant.  The  Operational  Flight  Trainer/ 
Weapons  Systems  Trainer's  integral  relationship  to 
flight  training  suggests  that  consideration  be 
given  to  remedial  action  relative  to  several 
problems  identified. 

(2)  Evaluative  Method:  The  heart  of  the  training 
effectiveness  methodology  was  assessment  of 
whether  the  following  criteria  were  met:  (a) 
prespecified  training  objectives,  (b)  structure 
and  control,  and  (c)  feedback  and  sequencing  based 
on  objective  performance  measurement. 

In  the  case  of  the  devices  under  study, 
application  of  the  criteria  seem  superficial  and 
unproductive.  The  real  question  of  interest  is 
efficiency,  not  effectiveness,  about  which  the 
method  does  not  make  claims.  An  important 
improvement  to  the  method,  if  the  checklist  is  to 
assist  in  evaluation,  is  a  description  of 
"accepted  training  principles"  and  an  improved 
method  for  determining  their  influence  on  device 
effectiveness. 
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The  authors  found  that  interview  data  obtained 
from  critiques  by  Phase  Officers  and  Instructor 
Pilots  were  more  informative  with  regard  to 
training  efficiencies,  problem  areas,  and 
implications  for  future  simulator  design,  than 
the  checklists . 
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1.  Author:  Caro,  Paul  W. 

2.  Title:  Equipment-Device  Task  Commonality  Analysis  and 
Transfer  of  Training. 

3.  Source:  Human  Resources  Research  Organization  ,  HUMRRO 
Technical  Report  70-7,  June  1970. 

4.  Topic  Keywords:  Task  Commonality  ; 

Transfer  of  training  ;  Flight  Simulation  . 

5.  Short  Summary:  A  commonality  analysis  was  performed  on 
a  training  device  that  was  previously  judged  deficient  in 
transferring  training.  This  analysis  establishes  the  source 
of  the  deficiency  as  a  lack  of  commonality  between  training 
device  controls  and  rotary  wing  aircraft  controls. 

6.  Devices: 

a.  Army  TH-13T  helicopter  (transfer  equipment) 

b.  1-CA-l  Fixed  Wing  Instrument  Trainer  (modified  for 
rotary  wing  instruction) 

7.  Institutions: 

a.  Sponsor:  Office,  Chief  of  Research  and 
Development,  Department  of  the  Army,  Washington,  DC 
20310. 

b.  Performing  organization: 

Human  Resources  Research  Organization  ,  300  North 
Washington  Street,  Alexandria,  VA  22314. 

8.  Type  of  Article:  Analytical. 

9.  Abstract:  The  implementation  of  procedures  for  task- 
commonality  analysis  arose  out  of  an  evaluation  of  a 
specific  training  device  used  in  Army  helicopter  training. 
The  device,  the  1-CA-l,  orginally  a  fixed-wing  instrument 
trainer,  had  been  modified  for  helicopter  instrument 
training,  but  had  been  found  deficient  in  producing  transfer 
of  training  to  the  rotary-wing  aircraft.  This  paper 
describes  the  method  used  to  analyze  the  deficiencies  with 
the  hope  of  altering  the  training  program  to  compensate  for 
them . 

Lists  were  made  both  of  hardware  stimuli  (displays)  and 
non-hardware  (environmental)  stimuli,  and  of  controls  in  the 
trainer  and  the  aircraft,  and  the  lists  were  compared.  An 
analysis  was  made  of  criterion  maneuvers  in  the  aircraft  to 
determine  what  displays  and  controls  (stimuli  and  responses) 
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were  used  to  execute  the  maneuvers.  Assessments  as  to  the 
realism  of  relevant  device  components  were  made  by 
helicopter  instructor  pilots. 

The  information  was  analyzed  on  the  basis  of  two 
tr an sfer-of- training  assumptions : 

a.  Postive  transfer  will  occur  when  both  stimuli  and 
responses  are  similar  in  the  training  situation  and  the 
criterion  situation; 

b.  Negative  transfer  will  occur  when  the  stimuli  are 
similar  in  the  training  and  the  criterion  situations, 
but  the  responses  to  the  similar  stimuli  are  different. 

It  was  judged  that  the  deficiency  in  the  trainer  resulted 
principally  from  a  lack  of  commonality  between  the  controls 
of  the  device  and  the  aircraft.  Of  seven  controls 
identified  as  important  in  the  performance  of  criterion 
maneuvers,  all  of  which  were  present  in  both  aircraft  and 
training  device,  three  were  rated  in  the  training  device  as 
unrealistic  in  either  direction  of  motion  or  effect  on 
displays.  In  addition,  an  eighth  control,  the  throttle,  was 
not  even  present  in  the  training  device. 

The  dissimilarities  in  the  controls  were  so  critical  as  to 
preclude  the  attempt  to  develop  a  new  training  program  using 
the  device.  It  was  felt  that  unacceptable  negative  transfer 
would  result  no  matter  what  revisions  were  made  in  the 
training  program.  Recommendation  was  to  use  the  device  as  a 
procedures  trainer,  and  only  in  adjunct  to  real-  aircraft 
flight  training. 
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1.  Author:  Caro,  Paul  W. 

2.  Title:  Transfer  of  Instrument  Training  and  the 
Synthetic  Flight  Training  System. 

3.  Source:  Human  Resources  Research  Organization  ,  HUMRRO 
Professional  Paper  7-27,  February  1972. 

4.  Topic  Keywords:  Instrument  Flight  Simulation  ; 

Transfer  of  Training  ;  Criterion  Based  Training  . 

5.  Short  Summary:  A  criterion-performance-based  training 
program  tailored  to  make  optimal  use  of  a  sophisticated 
instrument  flight  simulator  was  found  to  be  dramatically 
effective  in  teaching  student  pilots  the  instrument  phase  of 
helicopter  flight. 

6.  Devices: 

a.  Army  Synthetic  Flight  Training  System  (trainer) 

b.  UH-1H  helicopter  (test  device ) 

7.  Institutions: 

a.  Sponsor:  Office  of  the  Chief  of  Research  and 
Development,  Department  of  the  Army,  Washington,  DC 
20310. 

b.  Performing  Organization: 

Human  Resources  Research  Organization  ,  300  North 
Washington  Street,  Alexandria,  VA  22314. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  The  Subjects:  16 
candidates.  Officer  Rotary  Wing  Aviator  Course, 
transitioning  from  primary  contact  training  to 
instrument  flight  in  helicopter 

c.  Tests  or  Trials/Timing:  1  in-process  simulator 
test  (checkflight  in  simulator);  1  post-test 
(checkflight  in  aircraft) 

d.  Number  of  different  types  of  measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Time  to  criterion  and  amount  of  time  in 
check  r ide 

(2)  Checkride  grade  scored  by  instructor  pilot 


f.  Experimental  setting/training  context: 
Institutional,  hands-on 

g.  Statistical  methods:  Product  moment  correlation; 
others  not  specified 

h.  Variables  being  manipulated: 

(1)  Training  Device:  Synthetic  Flight  Training 
System 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified,  presumably  high 

(b)  Functional:  unspecified,  presumably  high 

(3)  Type  of  task/skill  required:  helicopter 
piloting,  operations,  cognitive,  psychomotor, 
perceptual,  procedural,  part-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  skill,  transition 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.: 
instr uc tor-managed 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  not  specified,  presumably 
intensive 

(2)  Features  used:  not  specified 
9 .  Abstract : 

a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
evaluate,  not  only  a  sophisticated  helicopter 
instrument  flight  training  simulator,  but  also  a 
newly-designed  training  program  which  was  tailored  to 
make  optimal  use  of  the  device.  Fundamental  to  the  new 
program  design  was  individualized  instructor-managed 
training,  where  student  advancement  through  the  program 
was  geared  to  the  achievement  of  criterion 
prof iciency--a  departure  from  the  lock-step  program 
which  was  standard  at  the  time  of  the  study. 

16  Army  helicoptr  trainees  who  had  completed  their 
primary  phase  of  helicopter  flight  training  were 
selected  to  take  the  experimental  course  of  instruction 
in  place  of  the  standard  instrument-flight-phase  course 
of  instruction.  Instead  of  the  standard  60  hours 
aircraft  time  plus  26  hours  simulated  training  on  the 
old  device,  the  subjects  received  a  mean  of  6  1/2  hours 
aircraft  time  and  42  hours  simulated  training  in  the 
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Synthetic  Flight  Training  System  (SFTS),  including  time 
for  checkrides. 


b.  Results: 

(1)  Key  Data: 

Training  Time  Requirements  in  the  SFTS  and  Grades 


of  Students  in 

Simulated 

Checkride  in  SFTS:  (in 

hours/minutes) 

Mean 

Std.  Deviation 

Training  Time 

40/28 

3/41 

Checkride  Time 

2/22 

/38 

Total  Time 

42/50 

3/47 

Checkride  Grade 

84.2 

7.6 

Aircraft  Familiarization 

and  Checkride  Time 

Requirements  and  Grades 

of  Students  in  the  UH-1 

Mean 

Std.  Deviation 

Training  Time 

4/12 

1/21 

Checkride  Time 

2/15 

/30 

Total  Time 

6/27 

1/31 

Checkride  Grade 

82.0 

6.2 

(2)  Verbal  Description: 

All  of  the  experimental 

students  passed  their  checkrides.  The  means  of 
their  performance  in  the  simulator  and  aircraft 
are  shown  above. 


c.  Author's  Conclusions:  The  study  provides  evidence 
that  simulators  can  be  used  as  effectively  with 
undergraduate  Army  trainees  as  with  highly  experienced 
commercial  pilots.  In  fact,  as  far  as  the  Instrument 
Phase  is  concerned,  the  Army  undergraduate  training  in 
the  experiment  was  significantly  more  effective  than 
the  conventional  training. 


"It  should  be  obvious  that  the  manner  in  which  the 
device  was  used  contributed  to  these  results  perhaps  as 
much  as  the  equipment  itself.  Undoubtedly,  had  any 
existing  synthetic  training  program  been  used,  much  of 
the  potential  effectiveness  of  the  SFTS  would  have  been 
lost,"  wrote  the  author. 


The  entire  training  program  was  cr i ter ion-per f ormance 
oriented,  and  so  unconventional  that  considerable  doubt 
was  expressed  by  experienced  aviators  concerning  its 
workability.  Their  doubts  have  been  resolved  by  the 
results  obtained. 


It  is  clear  that  military  pilot  training  organizations 
can  make  much  more  extensive  use  of  aircraft  simulators 
in  their  under-graduate  pilot  training  programs. 


improvements  were  made  in  the  simulator  itself  (to 
improve  its  physical  fidelity  and  to  quicken  its 
response),  and  major  changes  were  made  in  the  P.0.1. 

To  the  end  of  improving  the  P.O.I.  to  make  best  use  of 
all  T.D.'s  simulators  in  particular,  "an  explicit 
statement  of  minimal  trouble-shooting  standards  should 
be  made". 


b.  Results: 

(1)  Key  Data:  Overall  trouble-shooting  test 
proficiency  as  a  function  of  training  and  testing 
modes 

TOTAL  TEST  SCORES 


Training 

Hods 

Test  Mode 

Actual  Equip. 

Simulator 

Actual  N 

2  4 

28 

Mean 

23.56 

22.82 

SD 

1.86 

1.47 

Simulator  N 

29 

28 

Mean 

22.66 

22.86 

SD 

1.56 

1.53 

TOTAL  TIME  TO  COMPLETION  (MINUTES) 

Actual  N 

24 

18 

Mean 

50.21 

53.75 

SD 

9.14 

7.72 

Simulator  N 

29 

28 

Mean 

58.10 

55.93 

SD 

7.81 

9.66 

(2)  Verbal  Description:  The  only  significant 
differences  found  between  simulator- trained  and 
actual-  equipment- trained  students  on  any 
performance  measures  were  in  scores  and  time  taken 
in  the  trouble  shooting  test  when  done  on  actual 
equipment  (see  above),  and  these  differences  were 
very  slight.  "This  effect  is  minor...  and  may  be 
the  result  of  machine-specific  experience  rather 
than  qualitative  differences  in  training." 

In  the  attitude  questionnaire,  there  was  little 
difference  in  students'  attitudes  toward  the  two 
devices.  Instructors,  however,  were  more  critical 
of  the  simulator,  particularly  as  the  simulator 
was  deficient  in  response- time . 

A  life-cycle  cost  analysis  showed  the  simulator  to 
be  no  more  than  half  as  expensive  as  the  actual 
equipment  then  in  use  as  a  training  device. 

c.  Authors'  Conclusions:  At  the 

intermediate- introductory  level  of  training  where  this 
study  was  made,  no  practical  advantage  is  to  be  found 
in  the  use  of  actual  equipment  rather  than  the 
simulator.  The  authors  suggest  three  scenarios  for 
incorporation  of  the  simulator  in  the  P.O.I.:  (1)  to 

add  simulators  over  and  above  existing  actual  equipment 
training  devices;  (2)  to  combine  the  actual  equipment 
training  devices  and  the  simulators  in  a  complementary 
fashion;  < 3 )  to  replace  the  actual  equipment  T.D.'s 
with  simulators.  This  last  and  most  revolutionary 
option  would  serve  most  effectively  if  some 
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m.  Use  of  Instructional  Features: 

(1)  Intensity:  incomplete 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  The  objective  of  the  study  was  to  determine  the 
relative  effectiveness  of  a  3-dimensional  simulator  and 
the  6883  actual  equipment  test  station  on  dimensions  of 
instructional  efficiency,  attitudinal  acceptance,  field 
performance  of  simulator- trained  personnel,  and  life 
cycle  costs. 

Four  groups  of  28-30  aviation  maintenance  trainees  each 
were  trained  and  tested  either  on  the  simulator  or  on 
the  actual  equipment  which  was  the  standard  device  used 
in  the  curriculum.  One  group  was  both  trained  and 
tested  on  the  simulator;  another  was  both  trained  and 
tested  on  the  actual  equipment;  one  group  was  trained 
on  the  simulator  and  tested  on  the  actual  equipment; 
and  the  last  group  trained  on  the  actual  equipment  and 
tested  on  the  simulator. 

t)  •  In  addition  to  the  troubleshooting  proficiency  test  on 

the  devices,  the  trainees  were  tested  by  a  70-item 
pencil  and  paper  test  (short  answer  and  multiple 
choice).  Additional  measurements  were:  students' 
scores  during  subsequent  training  blocks;  and 
field/OJT  evaluation  by  supervisors  in  the  field. 
Furthermore,  students  evaluated  their  own  training  via 
questionnaires  immediately  after  the  experiment  and 
later  in  the  field;  instructors  evaluated  the  training 
via  questionnaire. 

Great  care  was  taken,  by  procedural  and  statistical 
methods,  to  weigh  and  where  possible  eliminate  the 
following  variables  as  significant  factors  in  the 
results:  training  sequence;  student  aptitude; 

student  sex  bias;  prior  adjustment  bias.  "Further 
correlation  analyses  showed  that  even  in  those  cases 
where  it  might  be  argued  that  a  slight  bias  existed, 
variables  were  not  meaningfully  related  to  performance 
on  the  trouble-  shooting  test." 
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(b)  29  same,  AET-trained  and  simulator- tested 

c.  Tests  or  Trials/Timing:  Two  immediate  post-tests; 
delayed  field/OJT  evaluation 

d.  Number  of  different  types  of  measures:  4 

e.  Description  of  Measurements  and  Ratings: 

(1)  Trouble-shooting  test,  hands-on  29-item  task 

(a)  total  score 

(b)  total  time 

(c)  degree  of  instructor  assistance  required 

(2)  Projected  Job  Proficiency  70-item  pencil  & 
paper  test 

(3)  Subsequent  training  instructional  block  scores 

(4)  Questionnaires  answered  by  field  supervisory 
personnel 

g.  Statistical  Methods:  Chi-Square;  ANOVA; 
Point-Biserial  Correlations;  Fisher's  Method  of 
Adj  ustment 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  6883-3D  three  dimensional  maintenance 
troubleshooting  simulator  for  F-lll 

(b)  Actual  Equipment  Training  Device  (AET) 

(2)  Fidelity  Levels: 

(a)  Physical:  for  simulator,  medium 

(b)  Functional:  for  simulator,  medium-high 
(slow  reaction) 

(3)  Type  of  Task/Skill  Required:  maintenance 
trouble-  shooting;  cognitive;  psychomotor; 
motor;  perceptual;  procedural;  part-task 

(4)  Task  Difficulty:  unspecified;  presumably 
medium-  high 

i.  Stage  of  Training:  introduction;  procedural; 
skill 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step  and 
instructor- managed 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  Reluctant  to  accept  simulator  as 
replacement  for  actual  equipment;  critical  of 
some  functional  deficiencies  (esp.  slowness  of 
simulator  response) 

(2)  Students:  good 
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1.  Authors:  Cicchinelli,  Louis  F. ,  Harmon,  Kenneth  R. , 

Keller,  Robert  A.,  &  Kottenstette ,  James  P. 

2.  Title:  Relative  Cost  and  Training  Effectiveness  of  the 
6883  Three-Dimensional  Simulator  and  Actual  Equipment. 

3.  Source:  Air  Force  Human  Resources  Laboratory  TR-80-24, 
September  1980. 

4.  Topic  Keywords:  Three-dimensional  Simulator  ; 

Training  effectiveness  ;  Maintenance  Training  ; 

Trouble-shooting  ;  Maintenance  Simulation  . 

5.  Short  Summary:  This  experiment  compared  test 
performance  and  other  proficiency  measures  from  groups  of 
maintenance  students.  These  groups  were  instructed  either 
on  actual  maintenance  test  station  equipment  or  a 
three-dimensional  simulator.  The  experiment  found 
significant  differences  in  only  one  task;  yet  using  actual 
equipment  was  twice  as  expensive  as  using  the  simulator. 

6.  Devices: 

a.  6383-3D  Three  dimensional  maintenance  simulator 
for  F-lll 

b.  6883  Actual  equipment  test  station  training  device 
(AET) 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  AFB ,  TX. 

b.  Performing  Organization: 

Denver  Research  Institute,  Social  Systems  Research  and 
Evaluation  Division,  University  of  Denver,  Denver,  CO 
80208 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  4 

b.  Description  of  Groups: 

( 1 )  Subjects: 

(a)  28  F - 1 1  Avionics  Maintenance  trainees, 
simulator- trained  and  Actual  Equipment- 
tested 

(b)  28  same,  simulator  trained  and  simulator 
tested 

(2)  Controls: 

(a)  30  same,  AET-trainea  ana  AET-tested 
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The  results  confirmed  the  cost-effectiveness  of 
designing  and  implementing  new  training  programs  to 
suit  the  capabilities  of  new  training  devices.  The 
training  program  used  with  a  device  is  at  least  as 
important  as  the  device  itself. 


b.  Results:  Four  experimental  subjects  in  a  newly 
designed  training  program  using  new  simulation  training 
device  reached  check  flight  criterion  in  aircraft  with 
considerably  less  hands-on  aircraft  instruction  time 
than  students  in  the  older  training  program  using 
either  old  or  new  equipment. 

Key  Data:  Training  Hours  Required  to  Reach  Criterion 

Aircraft  Device  Classroom 


Existing  Program 

60:00 

21:00 

Experimental 
Trainee  No. 

1 

33:55 

19:00 

Trainee  No. 

2 

3  4:05 

19:30 

Trainee  No. 

3 

34:30 

21:00 

Trainee  No. 

4 

35:10 

21:00 

90:00 

39:00 

39:00 

39:00 

39:00 


c.  Authors'  Conclusions:  The  training  program  used 
with  a  device  is  more  important,  from  the  transfer  of 
training  viewpoint,  than  the  device  itself. 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Off-the-shelf  fixed  wing 
synthetic  instrument  flight  procedure  trainer 
GAT-2;  however,  the  variable  of  interest  was  the 
training  program  itself,  rather  than  the  device. 

(2)  Fidelity  Levels:  Unspecified.  Presumed  high. 

(3)  Type  of  Task/Skill  Required:  Instrument 
Flight  Procedures 

(4)  Task  Difficulty:  Medium-High 

i.  Stage  of  Training:  transition  from  single-engine 
to  twin-engine  fixed  wing  aircraft 

j.  Trainee  Sophistication:  Medium-high 

k.  Incorporation  of  Device  into  P.O.I.:  Lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  Although  it  has  long  been  known 
that  factors  other  than  equipment/simulation  device 
similarity  impact  on  transfer  of  training,  study  of 
those  other  factors  is  seldom  sufficient.  When  a  new 
training  device  appears,  program  designers  and 
administrators  tend  to  rely  upon  the  device  itself  to 
assure  adequate  training. 

In  this  study,  a  follow-up  to  an  experimental  study 
which  established  the  training  effectiveness  of  the 
then-  new  GAT-2  Instrument  Flight  Simulator,  a  training 
program  was  designed  and  implemented  specifically  to 
exploit  the  advantages  of  the  new  device  over  an  old 
device  then  in  use  in  a  routine  school  program. 

The  new  program  has  25  hours  less  aircraft  time  (and 
also  quite  a  bit  less  classroom  time)  than  the  routine 
program,  while  training  device  time  on  the  new  device 
was  somewhat  above  training  device  time  on  the  old 
device  in  the  routine  program.  Four  subjects  trained 
experimentally  under  the  new  program  were  able  to  meet 
check  flight  criterion  after  completion.  The  savings 
in  real-aircraft  flight  training  due  to  the  new  program 
was  calculated  to  be  roughly  $1,000  per  trainee.  (Data 
on  the  relative  costs  of  the  old  versus  new  simulator 
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1.  Authors:  Caro,  Paul  W. ,  Isley,  Robert  N. ,  & 

Jolley,  Oran  B. 

2.  Title:  Research  on  Synthetic  Training:  Device 
Evaluation  and  Training  Program  Development  -  (Study  Two) . 

3.  Source:  Human  Resources  Research  Organization  Technical 
Report  73-20,  September  1973,  pp.  23-32. 

4.  Topic  Keywords:  Instrument  Flight  Simulator  ; 

Transfer  of  Training  ;  Program  Design  ;  Training  Program  . 

5.  Short  Summary:  This  report  on  the  implementation  of  a 
newly  designed  training  program  employing  a  new  simulator 
training  device  attempts  to  demonstrate  that  how  the 
training  program  uses  a  device  is  more  important,  from  the 
transfer  of  training  viewpoint,  than  the  device 
implementation  itself. 

6.  Devices:  Off-the-shelf  Fixed  Wing  Simulator  for 
instrument  flight  procedure  training. 

7.  Institutions: 

a.  Sponsor:  Office  of  the  Chief  of  Research  and 
Development,  Department  of  the  Army 

b.  Performing  Organization:  Human  Resources  Research 
Organization  Division  No.  6  (Aviation),  Fort  Rucker, 

AL  36362 

8.  Experiment: 

a.  Number  of  Groups:  1 

b.  Description  of  Groups: 

(1)  Subjects:  4  trainees,  Officer/Warrant  Officer 
Fixed  Wing  Aviator  Course 

(2)  Controls:  Post-test 

c.  Tests  or  Trials/Timing: 

a.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  training 
hours  to  criterion 

f.  Experimental  Setting/Training  Context: 

Institutional,  hands-on 

g.  Statistical  Methods:  not  specified 
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evaluation,  the  experimental  subjects  showed 
significantly  superior  performance  to  the  controls. 
Although  equipment  and  administrative  limitations 
precluded  clearly  valid  comparisons  with  any  group  who 
received  routine  training  on  the  old  simulator,  the 
performance  of  the  experimental  subjects  appeared 
roughly  at  least  equal  to  the  performance  of  those 
under  going  the  routine  training  on  the  old  simulator. 

The  study  concluded  that  the  design  and  implementation 
of  a  new  training  program,  to  exploit  advantages  of  the 
new,  higher-fidelity  training  device,  was  warranted. 
This  undertaking  is  reported  in  "Study  Two"  of  the  same 
Technical  Report,  which  is  summarized  in  another 
abstract  (same  title,  "Study  Two"). 

b.  Results:  Trainees  who  received  training  in  the  new 
device,  compared  with  control  group  trainees  (no 
simulation  training) ,  had  a  lower  attrition  rate  and 
were  more  likely  to  be  above-average  students, 
according  to  flight  instructor's  ratings.  In  check 
rides,  experimental  subjects  performed  better  at 
procedural  tasks  when  evaluated  subjectively  by  check 
pilots  and  when  scored  objectively  from  photographic 
records  of  check  ride. 

Key  Data:  Flight  Deficiency  Attrition 


Group 

Number 

Attrition 

Entering 

Training 

Stage 

One 

Stage 

Two 

Total 

Environmental 

24 

1 

1 

1 

Control 

16 

(p< .05) 

5 

0 

5 

c.  Authors'  Conclusions:  The  new  device  can 
contribute  to  the  effectiveness  of  twin-engine 
transition  and  instrument  training.  When  the  device  is 
used  with  a  training  program  employing  modern  training 
concepts,  significant  savings  to  the  Army  in  inflight 
training  time  can  be  obtained. 

There  has  been  no  evidence  presented  in  this  research 
that  synthetic  training  in  the  new  device  is  more 
effective  than  similar  training  in  existing  equipment. 
Nevertheless,  on  the  basis  of  its  greater  overall  task 
similarity  to  that  required  in  the  training  aircraft, 
one  would  predict  greater  transfer  from  the  new  device. 


73 


(2)  Supervisor  Ratings:  Check  Pilot  Checklists; 
Instructor  Questionnaires;  Instructor-assigned 
daily  flight  grades 

f.  Experimental  Setting/Training  Context:  School 

g.  Statistical  Methods:  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Off-the-shelf  fixed  wing 
synthetic  instrument  flight  procedure  trainer 
GAT- 2 

(2)  Fidelity  Levels:  Unspecified.  Presumed  high. 

(3)  Type  of  Task/Skill  Required:  Instrument 
Flight  Procedures 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  Transition  from  Single-Engine 
to  Twin-Engine  Fixed  Wing  aircraft 

j.  Trainee  Sophistication:  Medium  (had  single-engine 
flight  experience) 

k.  Incorporation  of  Device  into  P.O.I.:  Lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  discussed 

(2)  Features  used:  not  discussed 

9.  Abstract: 

a.  Study  Synopsis:  When  training  requirements  change, 
the  simulation  training  devices  in  use  and  their 
associated  training  programs  must  be  reevaluated  to 
assure  their  continued  effectiveness.  A  fundamental 
change  made  in  the  U.S.  Army  Aviation  School  in  FY  67, 
shifting  to  use  of  a  twin-engine  plane  as  the  principal 
instrument  training  aircraft,  afforded  the  opportunity 
to  test  the  effectiveness  of  a  new  training  device  in 
the  course  of  revising  then-existing  instrument 
training  programs. 

The  present  study  examines  the  substitution  of  the 
GAT-2  instrument  flight  simulator  in  place  of  the 
lower- f idel i ty  2B12A  device  then  in  use.  Experimental 
subjects  received  training  on  the  GAT-2;  controls 
received  no  simulation  training  whatsoever.  As 
measured  by  early  attrition  rates  and  by  checkflight 
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1.  Authors:  Caro,  Paul  W.  ,  Isley,  Robert  N. ,  & 

Jolley,  Oran  B. 

2.  Title:  Research  on  Synthetic  Training:  Device 
Evaluation  and  Training  Program  Development  -  (Study  One). 

3.  Source:  Human  Resources  Research  Organization  Technical 
Report  73-20,  September  1973;  pp.  1-22. 

4.  Topic  Keywords:  Instrument  Flight  Simulator  ; 

Transition  ;  Transfer  . 

5.  Short  Summary:  The  substitution  of  a  new  synthetic 
training  device  to  parallel  changes  in  equipment  (e.g., 
single-engine  to  twin-engine  aircraft),  can  be  made  in 
existing  training  program  with  acceptable  effectiveness. 

But  the  modification  of  the  training  program  is  indicated  in 
order  to  exploit  the  advantages  of  the  new  device. 

6.  Devices:  GAT-2  ,  off-the-shelf  Fixed  Wing  Simulator  for 
instrument  flight  procedure  training. 

7.  Institutions: 

a.  Sponsor:  Office  of  the  Chief  of  Research  and 
Development,  Department  of  the  Army 

b.  Performing  Organization: 

Human  Resources  Research  Organization  Division  No. 6 
(Aviation),  Fort  Rucker,  AL  36362 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  Two  Group  Post  Test 

b.  Description  of  Groups: 

(1)  Subjects:  24  Trainees,  Officer/Warrant 
Officer  Fixed  Wing  Aviator  Course,  with  mean  prior 
flight  experience  6.8  hours 

(2)  Controls:  16  same  with  mean  prior  flight 
experience  10.6  hours  (augmented  to  35  controls 
after  attrition  of  original  16) 

c.  Tests  or  Trials/Timing:  Post-test 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Performance  Measures:  Two  instrument-phase 
checkrides  with  scored  photographic  record; 
trainee  flight  records;  attrition  rate  in  early 
phases;  number  of  flight  hours  to  criterion 
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reduction  in  number  of  eliminations  due  to  flying 
deficiency  resulted  from  device  training. 

Device- trained  subjects  also  excelled  in  other 
performance  measures  during  early  stages  of  training. 
However,  after  the  first  solo  flight  on  real  aircraft, 
no  significant  differences  were  observed  between 
device- trained  subjects  and  control  groups,  on  any  of 
the  flight  performance  measures  used  in  this  study. 

The  difference  in  elimination  rates  took  place  mostly 
in  the  early  (Pre-Solo)  phase  of  flight  training.  The 
device  assisted  in  learning  of  gross  motor  skills; 
these  are  relatively  important  in  early  stages  of 
training,  but  at  later  stages  these  skills  decline  in 
importance  relative  to  more  specific  skills. 

Experimental  subjects  reached  a  Pre-Solo  checkride 
level  of  proficiency  somewhat  more  quickly  than  control 
subjects.  This  suggests  that  advancement  of  trainees 
according  to  proficiency  (rather  than  according  to 
experience)  could  lead  to  significant  savings  in  a 
flight  program. 

Experimental  subjects  were  subdivided  into  two  groups, 
of  which  one  received  a  mean  of  3.17  hours  on  the 
device,  the  other  a  mean  of  7.13  hours;  however,  no 
significant  differences  in  performance  measures  emerged 
between  these  two  groups.  This  indicated  a  practical 
value  in  investigating  the  effects  of  further 
reductions  of  device  training  time  to  find  an  optimum. 

b.  Results:  Simulator  device  training  reduced  the 
rate  of  eliminations  of  candidates  due  to  Flight 
Deficiency  from  30%  to  10%.  Also,  device-trained 
subjects  acquired  skills  necessary  for  solo  flight  with 
significantly  less  in-flight  training  than  controls. 

c.  Authors'  Conclusions:  "...  the  use  of  a 
helicopter  contact  flight  training  device  of  the  type 
used  in  this  study  could  result  in  a  significant 
reduction  in  elimination  rates  from  subsequent 
helicopter  flight  training  and  in  significant 
improvements  in  trainee  performance  early  in  such 
training. " 


(4)  Checkri.de  grades 

f.  Experimental  Setting/Training  Context: 
Institutional;  hands-on 

g.  Statistical  Methods:  ANOVA 

h.  Variables  Being'  Manipulated: 

(1)  Training  Devices:  Captive  Helicopter: 
Whirlymite  Helicopter  Trainer,  Model  DHT-1, 
one-man  helicopter  attached  to  Del  Mar  Ground 
Effect  Machine,  operating  in  real-world, 
out-of-door  environment 

(2)  Fidelity  Levels: 

(a)  Physical:  high 

(b)  Functional:  medium 

(3)  Type  of  Task/Skill  Required:  operations; 
psychomotor;  whole-task 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  introduction;  familiarization; 
transition 

1 .  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  Although  trainees  with  fixed-wing 
experience  are  rarely  eliminated  from  helicopter  flight 
training,  failure  rates  are  high  among  helicopter 
flight  trainees  in  general.  A  helicopter  flight 
simulator  providing  contact  aeronautical  experience 
gives  the  novice  the  opportunity  to  practice  contact 
flight  maneuvers  which,  at  his  proficiency  level,  would 
be  unsafe  in  an  aircraft.  The  device  selected  for  this 
study  differed  from  conventional  flight  training 
devices  in  operating  in  a  real-world,  out-of-door 
environment,  providing  visual,  auditory,  and 
proprioceptive  stimuli  associated  with  hovering  flight 
in  a  free-flying  helicopter. 

A  statistically  significant  (p=.006)  two-thirds 
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1.  Authors:  Caro,  Paul,  W.  &  Isley,  Robert 

2.  Title:  Helicopter  Trainee  Performance  Following 
Synthetic  Flight  Training. 

3.  Source:  Journal  of  the  American  Helicopter  Society, 

Vol.  11,  No.  3,  July  1966  pp.  38-44;  Human  Resources 
Research  Organization  Professional  Paper  7-66,  November 
1966  . 

4.  Topic  Keywords:  Contact  Aeronautical  Experience  ; 
Synthetic  Flight  Training  . 

5.  Short  Summary:  The  use  of  a  Synthetic  helicopter 
contact  flight  training  device  was  shown  to  be  associated 
with  a  significant  reduction  in  elimination  rates  from 
subsequent  helicopter  flight  training  and  in  significant 
improvements  in  trainee  performance  in  the  early  stages  of 
training. 

6.  Devices:  Captive  Helicopter;  Whirlymite  Helicopter 
Trainer,  Model  DHT-1 ,  one-man  helicopter  attached  to 
Del  Mar  Ground  Effect  Machine,  operating  in  real-world, 
out-of-door  environment. 

7.  Institutions: 

a.  Sponsor:  Department  of  the  Army 

b.  Performing  Organization: 

George  Washington  University  ,  Human  Resources  Research 
Office,  Division  6,  Fort  Rucker,  AL  36362 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  66  male  enlisted  volunteers, 

Warrant  Officer  Candidates,  Rotary  Wing  Aviator 
Course 

(2)  Controls:  66  same 

c.  Tests  or  Trials/Timing:  Post-test  and  in-process 
grades 

d.  Number  of  Different  Types  of  Measures:  4 

e.  Description  of  Measurements  and  Ratings: 

(1)  Rate  of  elimination  from  training 

(2)  Flight  i .me  to  checkride 

(3)  Daily  grades 

I  PREVIOUS  PAGE 
IS  LU  ANK 
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1.  Authors:  Cicchinelli,  Louis  F. ,  &  Harmon,  Kenneth  R. 

2.  "■  Training  and  Cost  Effectiveness  of  Two  Avionics 

Maintenance  Training  Simulators 

3.  Source:  Proceedings,  23rd  Annual  Conference  of 

the  Military  Testing  Association,  Arlington,  VA:  October 
25-30,  1981;  pp.  269-278 

4.  Topic  Keywords:  Maintenance  Training  ; 

Maintenance  Troubleshooting  ;  Simulation  ;  Fidelity  ; 

Low  fidelity  . 

5.  Short  Summary:  Two  studies  compared  two  simulators  of 
differing  fidelity  with  actual  equipment  used  in  maintenance 
training.  The  simulators  were  found  to  be  as  effective  as 
the  actual  equipment  and  far  less  expensive. 

6 .  Devices : 

a.  6883  Convertor/  Flight  Control  System  Maintenance 
Test  Station  for  the  F-lll 

b.  Three  Dimensional  6883  Convertor/  Flight  Control 
System  Simulator 

c.  Three  Dimensional  6883  C-FC  Simulator  including 
Patch-panel  Part-task  Train  s 

d.  Two  Dimensional  6883  Simulator  and  Part-task 
Trainer 

7 .  I nst i tut i ons  : 

a.  Sponsor:  Air  Force  Human  Resources  Laboratory, 
Lowry  AFB ,  CO. 

b.  Performing  Organization: 

Denver  Research  Institute,  Denver,  CO 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups: 

(1)  Study  One:  4 
(2}  Study  Two:  4 

b.  Description  of  Groups: 

(1)  Study  One 

(a)  28  F-ill  avionics  maintenance  trainees, 
simulator  trained,  simulator  tested 

(b)  28  same,  simulator  trained,  actual 
equipment  tested 


(c)  29  same,  actual  equipment  trained, 
simulator  tested 

(d)  30  same,  actual  equipment  trained,  actual 
equipment  tested 

(2)  Study  Two 

(a)  34  F-lll  avionics  maintenance  trainees, 
2-D  simulator  trained 

(b)  35  same,  3-D  simulator  trained 

(c)  13  same,  3-D  expanded  simulator  trained 

(d)  34  same,  actual  equipment  trained 

(Note:  test  modes  not  specified  for  Study 

Two  results) 

c.  Tests  or  trials/timing: 

(1)  Study  One:  2  post-tests 

(2)  Study  Two:  Pre-assessment  abilities  tests;  3 
performance  post-tests 

d.  Number  of  different  types  of  measures:  3  for 
hands-on  trouble-shooting  post-test;  2  additional  for 
other  tests 

e.  Description  of  Measurements  and  ratings: 

(1)  Hands-on  performance  test:  total  score 

(2)  Hands-on  performance  test:  time  to  completion 

(3)  Hands-on  performance  test:  degree  of 
assistance  required  for  completion 

(4)  Projected  Job  Proficiency  Test:  multiple 
answer 

(5)  (Study  Two  only)  Paper  and  pencil  test 

f.  Experimental  setting/training  context: 

Institutional,  hands-on 


g.  Statistical  methods:  One  way  ANOVA 


h . 


Variables  being  manipulated: 

(1)  Training  Device:  as  above,  section  6 

(2)  Fidelity  Levels: 


Actual  Equipment 
3-D  Simulator 
2-D  Simulator 


Physical  Functional 

High  High 

Med-high  (High) 

Med -low  (High) 

(assumed) 

(3)  Type  of  task/skill  required:  maintenance 
troubleshooting;,  cognitive,  psychomotor, 
procedural,  part-task  &  whole-task 

(4)  Task  Difficulty:  Medium-high 


l.  Stage  of  Training:  not  specified;  assumed 
introduction,  familiarization,  transition 
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j.  Trainee  sophistication:  not  specified;  assumed 
intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  not 
specified;  presumably  lock-step 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  presumably  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  Two  experiments  were  conducted  to 
compare  the  effectiveness  of  two  simulators  of 
differing  fidelity  with  the  actual  equipment  in  the 
training  of  maintenance  test  and  troublshooting  tasks. 
In  Study  One,  the  6883  Conver tcr/Flight  Control  System 
maintenance  test  station  for  the  F-lll  was  compared 
with  a  3-dimensional  simulator;  in  Study  Two,  these 
two  devices  were  compared  with  the  third  device,  a 
two-dimensional  simulator  (a  fourth  device  was  an 
expansion  of  the  3-D  simulator).  Subjects  were  F-lll 
avionics  maintenance  trainees,  115  in  Study  One  and  119 
in  Study  Two.  Measurements  in  Study  One  were  of 
performance  in  a  hands-on  post-test  and  in  a 
multiple-answer  Projected  Job  Proficiency  Test.  In 
Study  Two,  pre-assessment  ability  tests  were  given,  and 
a  third  paper-and-pencil  post-test  was  added. 

Cost  analyses  of  the  actual  equipment  and  the  3- 
dimensional  simulator  were  also  conducted. 

b.  Results: 

(1)  Key  Data: 

Training  Mode  (Study  Two) 


Hands 

AET 

-on  Score 

2-D 

3-D 

3-D/PPT 

Total 

X 

21.50 

22.53 

21.74 

21.77 

21.91 

N 

PJPT 

34 

Score 

34 

35 

13 

116 

X 

20.77 

20.58 

20.85 

21.23 

20.79 

N 

P&PTS 

35 

Scor  e 

36 

35 

13 

119 

X 

19.29 

19.47 

17.52 

21.23 

19.11 

N 

31 

36 

29 

13 

109 

(2)  Verbal  Description:  In  Study  One, 
simulator-trained  and  AET-trained  students  did  not 
differ  appreciably  with  respect  to  overall 
trouble-shooting  test  performance.  A  very  slight 
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advantage  in  test  accuracy  was  found  for  actual- 
as  opposed  to  simulator- trained  students  tested  on 
the  actual  equipment,  but  this  finding  was  not 
mirrored  using  completion  time  as  a  measure.  The 
Projected  Job  Proficiency  Test,  a  multiple  answer 
performance  test,  also  indicated  no  significant 
differences  among  groups  as  a  function  of  training 
mode . 


In  Study  Two,  a  preliminary  analysis  indicated 
that  there  were  no  significant  differences  in  the 
preassessment  scores  among  the  four  experimental 
groups . 

Performance  post-test  results  from  Study  Two  are 
shown  in  the  table  above.  When  all  four  levels  of 
training  were  submitted  to  one-way  analyses  of 
variance,  no  significant  differences  among  means 
were  found  for  any  of  the  three  measures.  It 
should  be  stressed  that  this  analysis  collapsed 
student  groups  across  testing  modes  and  that  being 
trained  and  tested  on  different  systems  may  result 
in  confounding  of  performance  measures  due  to 
unfamiliarity  with  the  equipment  used  for  testing. 
In  fact,  an  analysis  involving  only  those  groups 
of  students  who  were  trained  and  tested  on  the 
same  equipment  revealed  no  significant  differences 
in  performance  on  any  of  the  three  measures. 

Study  Two  analyses  are  preliminary,  and  a  final 
analysis  will  present  data  required  to  compare 
performance  as  a  function  of  training  equipment, 
when  only  the  actual  equipment  is  used  for 
testing. 


Cost  analyses  indicated  that  the  3-D  simulator  is 
at  most  half  as  expensive  as  the  actual  equipment. 
Cost  analysis  of  the  2-D  simultor  was  not  complete 
at  the  time  of  this  preliminary  report. 
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to  more  careful  control,  a  rigorous  cost  effectiveness 
analysis  of  simulation  training  is  not  possible. 


1.  Authors:  Cox,  John  A.,  Wood,  Robert  0.  Jr., 

Boren,  Lynn  M. ,  &  Thorne,  H.  Walter. 

2.  Title:  Functional  and  Appearance  Fidelity  of  Training 
Devices  for  Fixed-Procedures  Tasks 

3.  Source:  Human  Resources  Research  Organization  Technical 
Report  65-4,  June  1965. 

4.  Topic  Keywords:  Fidelity  ;  Fixed-Procedure  Task  ; 

Low  Fidelity  ;  Functional  Fidelity  . 

5.  Short  Summary:  Twelve  training  devices  of  varying 
levels  of  fidelity  were  compared  on  the  training  of  a 
procedural  task.  The  experiment  found  no  significant 
differences  in  effectiveness  due  to  the  fidelity  level  of 
the  training  device,  as  measured  by  transfer  to  the  highest 
fidelity  training  device  during  a  performance  test. 

6 .  Devices  : 

a.  High  Fidelity  Simulator  "Hot  Panel":  Tactical 
Equipment 

b.  High  Fidelity  Simulator  "Cold  Panel":  Tactical 
Equipment 

c.  High  Fidelity  Simulator  "Frozen  Panel":  Tactical 
Equipment 

a.  Cardboard  Panel  --  Entire  device  fabricatea  witn 
cardboard,  painted  to  resemble  device  a. 

e.  Photographic  Panel  —  a  full-size  B&W  photograph  of 
device  a. 

f.  Drawing  Panel  --  a  full-size  B&W  line  drawing  of 
device  a . 

g.  High  Fidelity  Housing  --  device  "b. "  with  replica 
of  device  "a."  housing 

h.  Box  Housin  --  device  "b. "  with  plywood  housing 

i.  Frame  Housing  --  device  "b.  "  with  wooden  frame 
housing 

j.  Full-Size  Panel  --  device  "f."  but  with  larger 
lettering 

k.  Half-Size  Panel  --  half-size  reproduction  of  de-ice 
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1.  Small  Panel  —  one-nineteenth  size  reproduction  of 
device  " j . " 

7.  Institutions: 

a.  Sponsor:  Office,  Chief  of  Research  and 
Development,  Department  of  the  Army  ,  Washington,  DC 
20310. 

b.  Performing  Organization:  Human  Resources  Research 
Office,  George  Washington  University  ,  Alexandria,  VA 
22314. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  16 

b.  Description  of  Groups: 

(1)  For  Studies  I-IV,  all  subjects  were  male  Army 
trainees  who  were  receiving  training  in  another 
specialty  than  the  one  being  studied;  to 
eliminate  confounds  due  to  prior  knowledge  of 
equipment,  GT  scores  were  as  closely  matched  as 
possible  among  all  groups. 

(a)  Study  One,  Functional  Fidelity:  3 
groups,  20  each 

(b)  Study  Two,  2-dimensional  v.s. 

3-dimensional:  4  groups,  20  each 

(c)  Study  Three,  Reduced  Housing,  3  groups, 
15  each 

(d)  Study  Four,  2-D  v.s.  3-D/Reduced 
Housing,  3  groups,  15  each 

(e)  Study  Five,  Size,  4  groups,  15  each 

(2)  Study  VI  subjects  were  actually  in  training 
for  the  specialty  including  this  equipment 
("Field"  study) 

(a)  35  controls 

(b)  36  subjects 

c.  Tests  of  Trials/Timing:  1  post-test  (run-through 
of  procedural  sequence) 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Number  of  correct  actions  taken  in  92-step 
procedure 

(2)  Time  to  complete  task 

f.  Experimental  Setting/Training  Context: 

(1)  Studies  I-V:  Laboratory,  hands-on 

(2)  Study  VI  ("Field"):  Institutional,  hands-on 
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g.  Statistical  Methods:  Analysis  of  covariance 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  above,  section  6 

(2)  Fidelity  Levels:  varied  from  low  to  high  in 
both  physical  and  functional  dimensions 

(3)  Type  of  Task/skill  required:  procedural, 
perceptual,  cognitive  (varying  degrees), 
psychomotor  (varying  degrees) 

(4)  Task  Difficulty:  Medium-low 

i.  Stage  of  Training: 

(1)  Studies  I-V:  procedural,  familiarization, 
skill 

(2)  Study  VI:  transition,  skill 

j.  Trainee  Sophistication: 

(1)  Studies  I-V:  novice 

(2)  Study  VI:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude  (Study  VI) 

(1)  Instructors:  generally  cooperative,  but 
doubtful  about  low-fidelity  devices 

(2)  Students:  not  discussed 

m.  User  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
evaluate  the  training  effectiveness  of  12  simulators 
which  ranged  in  fidelity  from  an  instrument  panel  to 
high  physical  and  functional  fidelity  down  to  a  line 
drawing  of  the  panel  which  measured  only  5"x7".  The 
task  chosen  for  study  was  a  long  (92-step)  but 
essentially  straightforward,  nonbranching  procedural 
sequence  of  low  difficulty.  Subjects  were  trained 
through  the  sequence  on  the  various  devices,  so  that  in 
the  course  of  five  laboratory  studies  all  devices  were 
used  by  at  least  one  group  once.  All  subjects  were 
subsequently  tested  on  the  high-fidelity  simulator 
which  had  served  as  a  trainer  for  some  of  the  groups. 
Subjects  were  matched  in  groups  according  to  GT  score 
as  closely  as  possible.  Analyses  of  covarience  were 
run  on  data  to  assure  that  aptitude  of  subjects  was  not 
a  determining  variable.  In  studies  I-V,  the  subjects 
were  purely  experimental,  being  enlisted  in  the  Army 
under  another,  somewhat  similar  specialty;  in  Study 
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VI,  called  by  the  authors  the  "Field  Study,"  subjects 
were  actually  undergoing  training  for  the  task  under 
study.  After  Study  VI,  participating  instructors  were 
interviewed  to  assess  their  attitudes  toward  the 
devices . 

b.  Results: 

(1)  Key  Data: 

Proficiency  Scores  of  Trainees 
By  Training  Group:  Study  II 


Hot 

Cold 

Photo 

Drawn 

Panel 

Panel 

Panel 

Panel 

(N=20 ) 

( N=20 ) 

( N=20 ) 

(N-20 ) 

Mean 

85.8 

82.4 

83.7 

82.0 

Median 

89.5 

83.5 

84.5 

86.0 

Proficiency  Scores  of  Trainees  by  Method 
and  Training  Group:  Study  V 


Method 

One 

Method 

Two 

Full-Size 

Half-Size 

Half-Size 

Small 

Panel 

Panel 

Panel 

Panel 

Mean 

81.9 

85.1 

83.7 

82.4 

Median 

84.0 

86.0 

83.0 

84.0 

Std.  Dev.  7.5 

5.1 

5.5 

6.9 

(2)  Verbal  Description:  No  significant 
differences  appeared  among  any  of  the  groups 
trained  on  the  various  devices,  in  the  laboratory 
or  in  the  field.  Results  from  Studies  I  and  V  are 
shown  above.  Remarkably,  the  small  line  drawing 
proved  as  effective  in  training  as  the  full-size 
panel . 

c.  Authors'  Conclusions:  When  men  are  being  trained 
to  perform  a  procedure  and  a  training  device  is  to  be 
used  as  a  method  of  training  them,  the  requirements  for 
functional  fidelity  in  the  device  are  quite  low. 
Limiting  factors  to  be  considered:  (1)  this  applies 
only  to  procedural  tasks;  (2)  the  training  device  does 
not  train  men  by  itself;  the  whole  training  process 
produces  the  effect.  Orientation,  techniques  of 
instructors,  and  psychological  atmosphere  all  effect 
acquisition  of  skill;  (3)  some  variations  occur  when 
verbal  signals  replace  visual  and  auditory  signals; 
while  these  variations  produced  no  significant 
differences  in  the  present  study,  changes  in  the 
training  environment  might  produce  differences  in  the 
effectiveness  of  devices  at  different  levels  of 
functional  fidelitv. 


The  size  study  extends  the  implications  of  the  other 
studies.  It  appears  that  small  pictures  or  drawings 
can  be  used  as  training  devices,  each  trainee  having 
his  own  device  from  which  to  study  and  perform.  The 
limitation  on  size  reduction  seems  to  be  only  that  the 
trainee  must  be  able  to  see  the  elements  on  the  device 
clearly  and  to  read  any  lettering. 

The  added  work  load  on  the  instructors  when  using  lower 
fidelity  devices  did  not  hamper  training  to  any 
measurable  extent.  The  field  study  data  showed  that 
instructors  were  able  to  perform  equally  well  under 
high  and  low  fidelity  conditions,  despite  the  lack  of 
confidence  expressed  by  some  instructors  in  the  low 
fidelity  devices. 


1.  Authors:  Crawford,  Alice,  Hurlock,  Richard, 

Padillo,  Robert,  &  Sassano,  Anthony 

2.  Title:  Low  Cost  Part-Task  Training  Using  Interactive 
Computer  Graphics  for  Simulation  of  Operational  Equipment 

3.  Source:  Defense  Technical  Information  Center  Technical 
Report  NPRDC  TR  TQ  76-46,  1976. 

4.  Topic  Keywords:  Computer-Aided  Instruction  ; 

Interactive  Graphics  ;  Computer  Graphics  ; 

Individualized  Training  ;  Part-Task  Training  ; 

Computer  Based  Training  ;  Learner  Control  . 

5.  Short  Summary:  A  computer-based  interactive  simulator 
with  program  versatility  is  a  cost-effective  low  fidelity 
alternative  to  higher  fidelity  devices  for  certain  tasks  in 
the  less  advanced  phases  of  training. 

6.  Devices:  Computer-Based  Simulator; 

High-Fidelity  Performance  Testing  Device 

7.  Institutions: 

f)  • 

a.  Sponsor: 

Navy  Personnel  Research  &  Development  Center,  San 
Diego,  CA 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment. 

a.  Number  of  groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  22  male  co-pilot  trainees 

(2)  Controls:  same 

c.  Tests  or  Trials/Timing:  Post-test 

a.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  two 
post-tests,  each  requiring  performance  of  two  tasks, 
consisting  of  6-  10  sequential  responses,  scored  by 
instructors 

f.  Experimental  Setting,  Training  context: 

g.  Statistical  Methoas:  F  test 

h.  Variables  Being  Manipulated: 
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(1)  Type  of  Training  Device:  PLATO  IV  Interactive 
Computer  Training  System,  with  8  1/2"  square 
plasma  display  panel  and  64-character  keyboard 
plus  touch  panels  with  256  programmable  areas  for 
entering  data;  testing  of  subjects  and  controls 
was  done  on  high-fidelity  simulator  which  was 
standard  training  device. 

(2)  Fidelity  Levels:  Low  to  Medium 

(3)  Type  of  Task:  Procedural  and  simple 
perceptual  motor  activities 

(4)  Task  Difficulty:  Unspecified,  presumably  only 
moderate 

i.  Stage  of  Training:  Copilot  training, 
familiarization  procedures 

j.  Trainee  Sophistication:  not  specified,  presumably 
novice  or  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  Attitude: 

(1)  Instructors:  not  specified 

(2)  Students:  Good.  After  testing,  most  trainees 
returned  to  practice  on  computer-based  simulator 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

9 .  Abstract : 


a.  Study  Synopsis:  Computer-Based  interactive 
simulation  of  equipment  using  a  versatile, 
variable-programmable  computer  graphics  display  (the 
PLATO  IV)  has  the  advantages,  over  high-fidelity, 
system-specific  simulators,  of  (1)  permitting  lesson 
materials  to  be  programmed  with  a  chosen  lesson 
strategy;  and  (2)  offering  more  training  on  different 
kinds  of  equipment  simply  by  changing  computer 
programs. 


Furthermore,  within  the  context  of  learning  specific 
tasks,  computer-based  simulation  (regardless  cf  type  of 
display)  can  provide  immediate  and  automatic  feedback 
and  monitoring  of  student  behavior. 

The  purpose  of  the  present  research  was  to  determine 
the  effectiveness  of  computer-based  graphic  simulation 
of  a  particular  piece  of  high-fidelity  training 
equipment,  in  the  teaching  of  certain  procedural  and 
simple  perceptual  motor  skills,  as  part  of  a  co-pilot 
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training  program.  An  additional  purpose  was  to 
evaluate  the  cost-effectiveness  of  the  specific 
computer  system  used,  the  PLATO  IV  developed  at  the 
University  of  Illinois;  and  a  cost  comparison  with 
conventional  training,  favorable  to  the  PLATO  IV,  is 
included  in  this  report. 

In  this  study,  3  hours  of  Computer-Based  Training  (CBT) 
was  given  to  the  experimental  subjects,  versus  3  hours 
of  conventional  workbook  study  assigned  to  the 
controls.  Subsequently,  on  a  high  fidelity  performance 
testing  device  (PT),  the  experimental  subjects 
performed  significantly  better  than  controls  on  an 
initial  test,  and  maintained  an  equal  superiority  over 
the  workbook  trained  students  on  a  second  test,  even 
after  both  groups  had  received  an  hour's  practice  on 
the  PT,  although  both  groups  improved.  Even  more 
startling,  the  performance  of  the  CBT  students  on  the 
first  test  (before  practice  on  the  PT)  was  superior  to 
the  controls'  performance  on  the  second  test  (after  an 
hour's  practice  on  the  PT) . 

Note:  Instructors  tested  on  the  PT  performed  far 

better  than  either  training  group,  showing  that  all 
student  learning  was  still  far  from  asymptotic. 

Although  training  time  on  the  high  fidelity  simulator 
(PT)  was  not  equal  to  training  time  on  the 
computer-based  simulator  (1  hr.  versus  3  hrs. ) ,  the 
authors  contend  the  computer-based  simulator  is  a 
cost-effective  alternative  to  the  higher- f idel i ty 
device  at  certain  tasks  in  the  less  advanced  phases  of 
training. 

b.  Results:  Comparison  of  Group  Performance  on  Tests 
1  and  2 

Test  1  Test  2 


Time 

No.  Tasks 

Time 

No.  Tasks 

(Min. ) 

Completed 

( Min . ) 

Completed 

Computer 

Based 

2.85 

1.68 

1  .88 

2.00 

Workbook 

4.44 

1.18 

3.32 

1  .73 

Students  trained  on  the  Computer-Based  Simulator  (CBT) 
completed  significantly  more  problems  and  took  less 
time  on  two  posttests  than  ncn-CBT  controls  who 
received  only  standard  workbook  instruction.  Moreover, 
after  an  hour's  acquaintanceship  with  the  high-fidelity 
performance  test  (PT),  the  CBT-trained  students 
maintained  their  superiority. 


c.  Authors'  Conclusions:  Computer-Based  Simulation 
provided  training  transferable  to  actual  equipment, 
without  degradation  due  to  absence  of  hands-on  practice 
with  actual  equipment.  On  certain  specific  procedural 
and  simple  perceptual  motor  skills,  practice  with  the 
high-  fidelity  simulator  could  be  eliminated  for  those 
receiving  Computer-Based  Simulation.  Considerable 
savings  might  be  realized  by  replacing  some  elements  of 
the  conventional  program  with  Computer-Based  Training. 
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c.  Authors'  Conclusions:  Computer-Based  Simulation 
provided  training  transferable  to  actual  equipment, 
without  degradation  due  to  absence  of  hands-on  practice 
with  actual  equipment.  On  certain  specific  procedural 
and  simple  perceptual  motor  skills,  practice  with  the 
high-  fidelity  simulator  could  be  eliminated  for  those 
receiving  Computer-Based  Simulation.  Considerable 
savings  might  be  realized  by  replacing  some  elements  of 
the  conventional  program  with  Computer-Based  Training. 


1.  Authors:  Cream,  Bertram  W. ,  Eggemeier,  F.  Thomas,  & 
Klein,  Gary  A. 

2.  Title:  A  Strategy  for  the  Development  of  Training 
Devices 

3.  Source:  AFHRL-TR-78-37 ,  August  1978;  Human  Factors  , 
1978,  20(2),  145-158 

4.  Topic  Keywords:  Flight  Simulation  ;  Task  Analysis  ; 
Training  Requirements  Analysis  ; 

Instructional  Systems  Development  . 

5.  Short  Summary:  A  methodology  for  training  device  design 
is  described.  The  results  of  an  empirical  study  evaluating 
a  device  designed  by  this  methodology  suggest  that  this 
methodology  produced  an  effective  and  well-  accepted  device. 

6.  Devices:  AC-130E  Gunship  Trainer 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB,  TX  78235 

b.  Performing  Organization:  Advanced  Systems 
Division,  Air  Force  Human  Resources  Laboratory, 
Wright-Patterson  AFB,  OH  45433 

8.  Experiment: 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  17  AC-130E  Gunship  students  (in 
three  position-designated  subgroups,  one  subgroup 
per  position) 

(2)  Controls:  27  same  (in  3  subgroups) 

c.  Tests  or  Trials/Timing:  Post-tests  (missions) 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  4-point 
Instructor  grade 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 

g.  Statistical  Methods:  Single-factor  ANOVA;  Duncan 
New  Multiple  Range  Test 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  AC-130E  Gunship  Trainer 

(2)  Fidelity  Levels:  not  specified 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual,  procedural, 
whole-task 

(4)  Task  Difficulty:  Medium-high  to  High 

i.  Stage  of  Training:  introduction;  skill 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  not  specified 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  positive  as  measured  by 
questionnaire 

(2)  Students:  very  positive  as  measured  by 
questionnaire 

9.  Abstract: 

a.  Study  Synopsis:  This  study  comprises  descriptions 
both  of  a  training  device  design  methodology,  and  of  an 
empirical  evaluation  of  a  trainer  which  was  designed 
according  to  that  methodology. 

The  methodology  assumes  that  a  simulation  training 
device  has  been  selected  as  a  medium.  The  roles  of 
three  groups — users,  training  psychologists,  and 
simulation  engineers — are  described  whereby  the 
psychologist  serves  as  interface  between  user  and 
engineer.  Together  with  the  user,  psychologists  put 
together  information  on  the  training  task  which  becomes 
the  input  to  the  engineer.  "Users. . .must  significantly 
influence  decisions  about  fidelity  and  capability.... 
The  psychologist  and  engineer  can  provide  the  user  with 
the  information  necessary  to  make  balanced  judgements 
regarding  fidelity,  cost,  and  training....  No  rigorous 
decision-making  procedures  have  been  developed  here. 

The  factors  in  the  cost  capability  tradeoff  are  not 
easily  quantified." 

The  heart  of  the  method  is  the  ranking  of  each 
feature — especially  the  task  and  sub-tasks  to  be 
trained —  along  three  dimensions:  criticality, 
frequency  of  performance,  and  difficulty  of 
performance,  with  criticality  the  most  heavily 
weighted.  These  rankings  serve  as  flexible  guides 
(rather  than  rigid  prescriptions)  to  user  and 
psychologist  as  to  what  features  to  include  and  the 
fidelity  level  of  each. 
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Performance  measures  and  crew  coordination  are  also 
discussed.  Practical  examples  substantiate  many  of  the 
principles  which  are  set  forth. 

The  trainer  developed  through  this  methodology  which 
was  evaluated  by  empirical  study  was  for  an  AC-130E 
gunship.  The  device  trained  a  crew  on  four  positions. 
Training  for  three  of  the  positions  was  evaluated  in 
this  experiment.  Training  for  control  subjects 
included  classroom  instruction  and  airborne  training. 
Training  for  experimental  subjects  consisted  of 
classroom  instruction,  training  with  the  new  device, 
and  aircraft  training.  There  were  9  control  subjects 
at  each  position;  experimental  subjects  numbered  6,  6, 
and  5  at  the  three  positions.  Performances  of  all 
subjects  in  ten  airborne  missions  subsequent  to 
ground-based  training  were  graded  by  instructors  on  a 
4-point  scale,  where  "0"  represented  inadequate,  "2" 
represented  qualified,  and  "3"  represented  completely 
qualified. 


b.  Results: 

(1)  Key  DatasNumber  of  Flying  Training  Missions  to 
Consistent  Criterion  for  Each  Subject  at  Each  of 
Three  Positions: 

POSITION  1  POSITION  2  POSITION  3 


Experi¬ 

Control 

Experi 

mental 

mental 

2 

5 

4 

2 

7 

4 

3 

7 

5 

3 

9 

5 

4 

9 

5 

6 

9 

7 

10 

10 

10 

Control 

Experi¬ 

mental 

Control 

4 

2 

4 

6 

3 

5 

6 

5 

5 

7 

5 

5 

7 

7 

6 

7 

7 

8 

7 

8 

7 

9 

8 

X=3 . 3  X=8 . 4  X=5  X=6 . 8  X=4 . 4  X=6 

(2)  Verbal  Description:  The  Duncan  New  Multiple 
Range  Test  indicated  that  the  Position  1 
experimental  group  differed  significantly  from  its 
control  (p<0 . 01) ,  and  that  the  Position  2  and  3 
experimental  groups  differed  from  their  controls 
(p<0 . 05) .  Experimental  groups  reached  consistent 
criterion  with  less  airborne  training  time  than 
did  control  groups  at  each  of  the  three  positions. 


Also,  at  each  position,  a  greater  percentage  of 
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subjects  trained  with  the  device  received 
"completely  qualified"  ratings  than  did  control 
subjects.  This  proficiency  effect  was  most  marked 
in  the  Position  1  group  and  represented  only  a 
consistent  trend  in  the  other  two  groups. 

Also,  a  questionnaire  indicated  high  acceptance  of 
the  device  among  all  users. 

c.  Authors'  Conclusions:  "It  can  be  concluded  that 
design  of  a  training  device  according  to  the  strategy 
(described)  did  produce  an  effective  device  which  was 
well  accepted  by  the  users." 


1.  Author:  Crosby,  John  V. 

2.  Title:  Cognitive  Pretraining:  An  Aid  in  the  Transition 
from  Instrument  to  Composite  Plying 

3.  Source:  AFHRL-TR-77-62 ,  October  1977 

4.  Topic  Keywords:  Cognitive  Pretraining  ; 

Flight  Training  ;  Schema  ;  Stimulus-Response  . 

5.  Short  Summary:  Cognitive  pretraining  appears  to  be 
effective  in  preparing  student  pilots  to  make  the  transition 
from  basic  instrument  flight  simulation  to  real  aircraft 
visual  flight. 

6.  Devices:  Photographs 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Department  of  Psychology, 
Arizona  State  University 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  12  Air  Force  undergraduate  pilot 
training  students 

(2)  Controls:  12  same 

c.  Tests  or  Trials/Timing:  Post-test 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Laboratory  measurement  of  perceptual  accuracy 
of  view 

(2)  Flight  maneuver  simulated  checkride, 
instructor-  rated  on  scale  of  1  -  12 

f.  Experimental  Setting/Training  Context: 

Institutional;  hands-on 

g.  Statistical  Methods:  Multivariate  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  photographs 

(2)  Fidelity  Levels: 
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(a)  Physical:  Medium-low 

(b)  Functional:  Low 

(3)  Type  of  Task/Skill  Required:  operations; 
perceptual 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  transition;  post-basic 
instrument;  pre-flight 

j.  Trainee  Sophistication:  novice  with  less  than  50 
hours  flying  time 

k.  Incorporation  of  Device  into  P.O.I.:  self-paced 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  discussed 

(2)  Features  used:  not  specified 

Abstract: 

a.  Study  Synopsis:  The  USAF  Undergraduate  Pilot 
Training  Program  at  Williams  AFB,  AZ,  is  designed  to 
train  students  to  fly  high-performance  jet  aircraft. 
Central  to  the  training  philosophy  is  the  concept  that 
students  acquire  and  develop  good  judgement  skills, 
showing  adaptability  and  flexibility  of  performance, 
rather  than  fixed  stimulus-response  connections.  In 
line  with  this  concept,  an  experiment  was  conducted 
using  trainees  to  examine  the  benefits  of  providing 
students  with  cognitive  pretraining  of  visual  flight 
situations  following  ground-based  simulated  instrument 
flight  training  and  prior  to  the  actual  flying  contact 
phase  of  training.  This  cognitive  pretraining,  it  was 
hypothesized,  would  assist  students  in  the  usually 
difficult  transition  from  basic  instrument  flying  to 
the  situation  where  80%  of  the  information  is  obtained 
from  non-instrument  sources. 

A  group  of  12  subjects  was  provided  with  the  cognitive 
pretraining  prior  to  flight  training,  and  the  group's 
performance  on  (1)  laboratory  tests  of  perceptual 
accuracy  of  view  from  cockpit,  and  (2)  a  high-fidelity 
simulator  test  on  flight  maneuvers,  was  compared  with 
the  performance  of  a  12-member  control  group  who 
received  no  cognitive  pretraining.  The  cognitive 
pretraining  materials  consisted  partly  of  photographic 
representations  of  the  cockpit,  showing  visual 
orientation  references  that  would  be  used  in  visual 
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flight  (these  photographs  might  be  considerd  to  be  low 
fidelity  training  devices) . 

The  performance  of  the  experimental  group  was 
significantly  superior  to  the  control  group's,  and  in 
the  cognitive  laboratory  test  portion  of  testing  was 
equal  and  in  one  parameter  even  superior  to  that  of  a 
group  of  12  experienced  Instructor  Pilots  who  were  used 
as  a  further  control. 

The  experiment  appeared  to  demonstrate  the  value  of 
cognitive  pretraining  in  the  development  of  flexible 
and  adaptive  skills  in  flight  training.  The  concept  of 
schemata  (for  which  the  author  proposed  Evans' 
provisional  definition  "a  schema  is  a  characteristic  of 
some  population  of  objects.  It  is  a  set  of  rules  which 
would  serve  as  instructions  for  producing,  in  essential 
aspects,  a  population  prototype  and  object  typical  of 
the  population....")  is  helpful  in  examining  this 
training  phenomenon. 

b.  Results:  Subjects  who  were  given  cognitive 
pretraining  performed  better  than  controls,  both  on 
medium-fidelity  laboratory  trials  of  visual  pitch  and 
bank  discrimination  for  cockpit,  and  in  flight 
maneuvers  executed  in  a  high-  fidelity  simulation 
testing  device. 

c.  Author's  Conclusions:  Cognitive  pretraining 
appears  to  offer  a  sound  and  economical  approach  to 
many  aspects  of  flying  training  research.  The  concept 
of  a  schema  places  less  emphasis  on  fixed 
stimulus-response  chains  and  presumably  more  emphasis 
on  the  development  of  flexible  and  adaptive  skills. 
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1.  Authors:  Crosby,  John,  Pohlmann,  Lawrence, 

Leshowitz,  Barry,  &  Waag,  Wayne 

2.  Title:  Evaluation  of  a  Low  Fidelity  Simulator  (LFS)  for 
Instrument  Training 

3.  Source:  Technical  Report,  AFHRL-TR-78-22 ,  July  1978 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Temporal  Duration  of  Effect  ;  Low  Fidelity  . 

5.  Short  Summary:  Groups  receiving  "Low  fidelity  simulator 
pretraining"  showed  positive  transfer  early  in  pilot 
training.  After  one  month  of  academic  and  higher  fidelity 
simulator  training,  however,  the  pretraining  showed  no 
effect. 

6.  Devices:  Low  Fidelity  Computer  Monitored 
Instrument  Flight  Simulator 

7.  .Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Arizona  State  University/ 
Flying  Training  Division,  Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  7  Undergraduate  Pilot  Trainees,  sex 
unspecified 

(2)  Controls:  7  same 

c.  Tests  or  Trials/Timing:  Pretest  simulator  sortie, 
post-  test  simulator  sortie,  post-test  contact  flight 
evaluation 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Performance  Measures:  Evaluation  sortie  in 
Advanced  Simulator  for  Pilot  Training  (ASPT) 
before  and  after  normal  T-4  instrument  training; 
mean  trials  to  criterion  for  T-4  instrument 
training;  mean  trials  to  criterion  for  subsequent 
basic  real  aircraft  contact  training 

(2)  Supervisor  Ratings:  Not  discussed 
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f.  Experimental  Setting/Training  Context: 

Undergraduate  Pilot  Training  Program 

g.  Statistical  Methods:  Not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Device:  Low-Fidelity 
Computer-Monitored  Instrument  Flight  Simulator: 
Joystick  mounted  to  straight-back  chair,  responds 
to  x-y  pilot  input,  simulates  pitch  and  roll 
controls;  adjacent  lever  simulates  throttle. 
Abbreviated  instrument  panel. 

(2)  Fidelity  Level:  Low 

(3)  Type  of  Task/Skill  Required:  Basic  Instrument 
Flight  Maneuvers 

(4)  Task  Difficulty:  not  evaluated 

i.  Stage  of  Training:  Pre-Flight,  Pre-High-Fidelity 
Simulator 

j.  Trainee  Sophistication:  Moderate  (both  groups  had 
similar  light  aircraft  flying  experience;  no  subjects 
had  jet  aircraft  flying  experience) 

k.  Incorporation  of  Device  into  P.O.I.:  Not  discussed 

l.  User  Acceptance  or  Attitude:  Not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  Not  specified 

(2)  Features  used:  Not  specified 

9.  Abstract: 

a.  Study  Synopsis:  An  inexpensive  Low  Fidelity 
Simulator  (LFS)  was  developed  and  used  to  appraise  the 
transfer  of  simulator  training  to  (1)  more  advanced 
simulator  training,  and  (2)  aircraft.  The  relevance  of 
the  LFS  to  real  flying  experience  was  confirmed  by  a 
preliminary  study. 

Two  groups  of  7  trainees  each  were  tested  on  a  full- 
mission  simulating  device  before  beginning  normal  basic 
academic  and  basic  (T-4)  instrument  training.  They 
were  again  tested  on  the  sortie  after  a  month  of 
academic  and  basic  instrument  training  and  before 
contact  flight  training.  The  experimental  group's 
training  differed  from  the  control  in  receiving  10 
hours  LFS  basic  instrument  maneuver  flying  instruction 
prior  to  the  first  evaluation  sortie  and  other, 
subsequent,  instruction.  The  control  groups  had  no  LFS 
training  at  any  time.  While  the  experimental  group 
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performed  significantly  superior  to  the  control  groups 
on  the  first  evaluation  sortie  (the  control  group 
having  received  no  training  whatsoever  at  this  point) , 
it  failed  to  excel  on  the  second  evaluation  sortie  a 
month  later — after  both  groups  had  received 
approximately  35  hours  medium-high-fidelity  basic 
instrument  (T-4)  training  and  28  hours  academic 
procedures  training;  and' the  experimental  group  failed 
subsequently  to  excel  the  controls  in  contact  flight. 

In  fact,  among  the  experimental  subjects,  there  was 
little  difference  between  their  performance  on  the 
first  and  second  evaluation  sorties.  This  suggests 
that  the  LFS  training  produced  a  "ceiling  effect"  on 
the  subjects,  such  that  subsequent  higher  fidelity 
training  failed  to  improve  their  skills.  Demand 
effects  and/or  ability  of  IP's  during  LFS  training  may 
have  influenced  this  result  as  much  as  the  simulator 
device  itself. 

Why  simulator-trained  students  (subjects  as  well  as 
controls)  required  a  high  number  of  trials  to  acquire 
skills  in  subsequent  contact  maneuvers  is  discussed  in 
Section  9c,  "Authors'  Conclusions"  below.  The 
explanations  for  these  shortcomings  can  be  briefly 
summarized : 

(1)  The  LFS  does  not  faithfully  reproduce  the 

"feel  of  the  aircraft" — a  consequence  of  low 

fidelity ! 

(2)  Inappropriate  simulator-acquired  skills  may 

impede  acquisition  of  contact  maneuver  skills. 

b.  Results:  On  the  initial  evaluation  sortie  before 
normal  T-4  instrument  and  academic  training,  the  Low 
Fidelity  Simulator- trained  group  performed 
significantly  better  than  controls  across  all 
maneuvers,  and  during  T-4  training  required  fewer 
trials  to  criterion.  After  T-4  training,  however,  and 
likewise  during  T-37  real  aircraft  training,  no 
differences  were  found  between  experimental  and  control 
groups. 

c.  Authors'  Conclusions:  Low-fidelity  simulator 
training  prior  to  onset  of  academic  and  higher-fidelity 
simulator  instruction  did  not  produce  significant 
differences  between  experimental  and  control  subjects 
during  subsequent  phases  of  training  (basic  instrument 
and  basic  contact) .  The  evidence  supports  the 
explanation  that  the  LFS  training  brought  the 
experimental  trainees  early  to  a  performance  ceiling 
beyond  which  they  were  less  likely  to  progress.  The 
slowness  of  subjects  to  learn  to  execute  maneuvers  in 


real  aircraft  can  be  ascribed  to  the  lower  level  of 
stress  and  the  lack  of  "feel  of  aircraft"  in  simulator. 
Furthermore,  the  "unlearning"  of  possible  inappropriate 
skills  "overlearned"  in  the  simulator  may  impede 
contact  maneuver  performance. 

Lastly,  the  initial  high  positive  transfer  of  simulator 
learning  in  the  experimental  group  may  be  due  to  the 
demand  effects  and/or  instructional  ability  of  the 
IP's.  Transfer  of  training  studies  should  investigate 
the  instructive  role  of  student,  instructor,  and  T.D. 
as  a  system  rather  than  focusing  exclusively  on  the 
simulator. 
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post-test  (transfer,  on  shipboard) 

d.  Number  of  Different  Types  of  Measures:  17 

e.  Description  of  Measurements  and  Ratings: 

(1)  Team  Measures: 

(a)  Average  aircraft  separation  error 

(b)  Time  to  complete  Recovery  of  11  aircraft 

(c)  Communications  noise  level 

(2)  Subteam  Measures: 

(a)  Aircraft  position  at  one  mile:  average 
azimuth  error 

(b)  Aircraft  position  on  one  mile:  average 
glideslope  error 

(c)  Communications  efficiency 

(d)  Communications  noise  level 

(3)  Individual  Measures: 

(a)  Radar  contact  not  announced  to  pilot 

(b)  Use  of  primary  approach  frequency  within 
1/2  mile  of  ramp  (inappropriate  use) 

(c)  Number  of  conflictions  (inadequate 
spacing  between  aircraft) 

(d)  Deviation  from  schedule  ramp  time  for  1st 
aircraft 

(e)  Marshall  information  broadcast:  t 

completeness  and  accuracy 

(f)  EAT  deviations  beyond  EAT  plus  or  minus 
.25  minutes  not  acted  upon 

(4)  Controller  Self  Evaluations 

(5)  Instructor  Evaluations 

(6)  LSO  Evaluations  of  CATCC  Performance 

(7)  Pilot  Evaluations 

f.  Experimental  Setting/Training  Context: 

Institutional,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  CATCC:  Carrier  Air  Traffic  Control 
Center,  a  multi-station  team  facility  on 
shipboard 

(b)  CATCC  Trainer:  a  land-based  simulator  of 
the  CATCC 

*  Note:  Chief  variable  was  time  on  device. 

(2)  Fidelity  Levels: 

(a)  Physical:  medium-high 

(b)  Functional:  medium-high 

(3)  Type  of  Task/Skill  Required:  aircraft 
piloting  operations;  cognitive;  perceptual; 
procedural;  whole-task 
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1.  Authors:  Finley,  Dorothy  L. ,  Rheinlander,  T.  W. , 
Thompson,  E.  A.,  &  Sullivan,  D.  J. 

2.  Title:  Training  Effectiveness  Evaluation  of  Naval 
Training  Devices.  Part  I.  A  Study  of  the 
Effectiveness  of  a  Carrier  Air  Traffic  Control  Center 
Training  Device 

3.  Source:  N AVTRAEQU I PCEN  70-C0258-1,  August  1972; 
National  Technical  Information  Service  AD  751  556 

4.  Topic  Keywords:  Carrier  Air  Traffic  Control  ; 
Analytical  Models  ;  Training  Effectiveness  ; 

Simulation  . 

5.  Short  Summary:  Evaluation  of  a  Carrier  Air  Traffic 
Control  Center  simulator  finds  it  effective  in  training 
teams  in  the  task  of  landing  aircraft. 

6.  Devices: 

a.  CATCC :  Carrier  Air  Traffic  Control  Center,  a 
multi-  station  team  facility  on  shipboard 

!)• 

b.  CATCC  Trainer:  a  land-based  simulator  of  the  CATCC 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization: 

Bunker  Ramo  Electronic  Systems  Division,  Westlake 
Village,  CA 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2  (roughly) 

b.  Description  of  Groups: 

(1)  Subjects:  "Ship  One" — 9  trainees  with  1-3 
weeks  training  on  the  simulator 

(2)  Subjects:  "Ship  Two" — 6  trainees  with  5  weeks 
training  on  simulator 

Note:  The  teams  representing  "Ship  One"  and  "Ship  Two" 

were  composed  of  various  combinations  of  the  trainees 
in  each  group,  and  also  included  at  least  one  person 
who  had  received  no  simulator  training,  and  another 
highly-  experienced  controller. 

c.  Tests  or  Trials/Timing:  In-process  trials; 
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provided  by  instructors  can  be  used  to  identify  problem 
areas  and  to  identify  the  resistance  which  may  be  met 
to  the  proposed  use  of  simulators  in  place  of  actual 
equipment. 


survey  of  training  equipment  problem  areas  and  new 
simulator  requirements  for  maintenance  courses. 

b.  Results: 

(1)  From  the  first  questionnaire:-  most 
instructors,  regardless  of  the  type  of  equipment 
covered  in  their  course  or  the  level  at  which  the 
course  was  conducted,  answered  the  questionnaire 
in  a  similar  fashion.  In  most  courses  actual 
equipment  trainers  were  employed  and  instructors 
preferred  it  that  way  even  though  many  complained 
about  the  reliability  of  the  equipment.  Many 
instructors  expressed  a  willingness  to  use  less 
expensive  training  devices  and  media  if  they  were 
provided  with  them  and  if  they  were  convinced  of 
their  effectiveness. 

(2)  From  the  second  questionnaire:  Few 
instructors  registered  complaints  about  the 
training  equipment  at  their  disposal.  However,  a 
substantial  minority  did  report  that  they  had 
problems  with  the  reliability  of  their  trainers. 

Of  the  36  top  candidate  equipments  for  simulation, 
19  were  identified  by  Lowery  AFB  instructors  known 
to  be  involved  with  or  knowledgeable  about 
simulation  efforts  related  to  the  test  bench.  An 
additional  9  of  the  top  36  were  identified  by 
instructors  at  Keesler  AFB,  most  of  whom  were 
involved  with  or  knowledgeable  about  an  on-going 
simulation  effort  at  that  base.  Of  the  six  high 
priority  simulation  candidates  identified  by 
Chanute  AFB  instructors,  three  of  these  pertained 
to  flight  simulators  of  the  type  for  which 
simulator  specifications  were  being  developed. 

c.  Authors'  Conclusions:  With  regard  to  use  of  new 
devices,  and  especially  low  fidelity  simulators, 
instructors  tend  to  be  conservative  in  that  they  prefer 
to  use  proven  instructional  devices  and  techniques  with 
which  they  are  familiar.  This  suggests  that 
instructors  may  be  good  information  sources  with 
respect  to  the  effectiveness  of  training  devices  and 
techniques  but  that  they  normally  would  not  be  good 
sources  of  information  about  where  and  under  what 
conditions  state-of-the-art  training  devices  might  be 
employed.  Instructors  tend  to  recommend  the  use  of 
simulators  to  the  extent  that  they  are  familiar  with 
simulators  and  how  they  can  be  effectively  employed  to 
teach  maintenance  on  the  type  of  equipment  covered  by 
their  course. 

With  respect  to  instructor  evaluation  of  training 
equipment  in  use  at  the  time,  information  and  opinions 
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1.  Authors:  Fink,  C.  Dennis,  &  Shriver,  Edgar  L. 

2.  Title:  Maintenance  Training  Simulators  at  Air  Force 
Technical  Training  Centers:  Present  and  Potential  Use 

3.  Source:  AFHRL-TR-78-77 ,  December  1978 

4.  Topic  Keywords:  Maintenance  Training  Simulation  . 

5.  Short  Summary:  A  study  of  the  need  for  simulators  in 
Air  Force  maintenance  training  through  questionnaires 
administered  to  instructors,  finds  the  questionnaire 
instrument  of  some  usefulness,  primarily  limited  to 
identifying  problem  areas  calling  for  the  intervention  of 
simulation  and  training  experts. 

6.  Devices:  Many  actual  equipment  devices  are  listed  as 
prime  targets  for  simulation. 

7.  Institutions: 

a.  Sponsor:  HQ  AFHRL  Air  Force  Human  Resources 
Laboratory  (AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Kinton,  Inc.,  1500  North 
Beauregard  Street,  Suite  205,  Alexandria,  VA  22311 

8.  Type  of  Article:  Survey 

9.  Abstract: 

a.  Study  Synopsis:  This  study,  employing 
questionnaires  administered  to  98  Air  Force  maintenance 
instructors  as  its  instrument  of  evaluation,  had  two 
quite  different  but  interdependent  objectives: 
primarily,  to  identify  the  present  and  potential  need 
for  maintenance  simulators  in  support  of  training 
conducted  at  Air  Force  Technical  Training  Center; 
secondarily,  to  investigate  the  usefulness  of  survey 
procedures  for  identifying  training  equipment 
simulation  candidates — that  is,  to  assess  the  value  of 
the  questionnaire  instrument  itself.  A  third  and 
lesser  objective  was  to  obtain  opinions  about  the  use 
of  low  to  medium  cost-fidelity  training  devices.  One 
hundred  different  courses  were  surveyed,  including 
equipment  in  the  following  areas:  electronic, 
electro-mechanical ,  pr ec ison/me a sur ing , 
electrical/telecommunications,  engines  (aircraft), 
hydraulic,  and  miscellaneous  others. 

Two  types  of  questionnaires  were  used:  a  survey  of 
instructor  opinions  regarding  use  of  low  and  medium 
cost/fidelity  training  devices  and  simulators,  and  a 


develop  a  tendency  for  greater  variability,  and  this 
carries  over  into  the  transfer  task  and  also  into 
post-evaluation  retention  performance. 

The  study  results  demonstrate  that  simulation  fidelity 
can  be  degraded  by  using  rigid  coefficients  in 
programmed  flight  equations  and  still  be  an  effective 
condition  for  training.  One  can  infer  from  this 
finding  that  high  engineering  fidelity  is  not  a 
necessity. 


Comparisons  Between  Average  Performances  of 
Experimental  Group  {Least  Squares)  Transfer  Trials 
with  Control  Group  (Flexible  Data)  on  Comparable 
Trials 


Control 

Expermt. 

Ratio 

Prob. 

Parameter 

Group(C) 

Group(E) 

E/C 

Phase 

Altitude  Error 

208.00 

541.00 

2.60 

.02* 

Heading  Error 

17.50 

25.20 

1 .44 

.20 

Climbing 

Turn 

Fore/Aft 

Stick  Var. 

2.50 

5.90 

2.36 

.03* 

Lateral 

Stick  Var. 

.37 

.85 

2.30 

.03* 

Altitude  Error 

201.00 

176.00 

.87 

.45 

Heading  Error 

21.10 

29.00 

1.37 

.25 

Level  Turn 

Fore/Aft 

Stick  Var. 

2  .60 

2.50 

.96 

.45 

Lateral 

Stick  var. 

.43 

.93 

2.16 

.05* 

*  Exceeds  .05  Significance  Level 


(2)  Verbal  Description:  During  both  transfer  and 
retention  testing,  significant  differences  did  not 
appear  between  controls  and  subjects  trained  in 
the  "rigid  airframe"  condition.  Between  the 
controls  and  the  subjects  trained  on  the 
least-square  condition,  significant  differences  in 
system  output  did  appear  in  altitude  error  only. 
However,  data  on  pilot  input  showed  that  the 
least-square  group  had  to  work  much  harder  (i.e., 
there  was  more  stirring  of  the  stick)  to 
accomplish  the  same  output  result. 

Statistical  analysis  of  the  pilot  questionnaire 
was  not  performed  due  to  sample  size.  However,  no 
correlation  in  the  questionnaire  results  was 
observed  to  suggest  that  the  pilots  had 
discriminated  between  the  different  levels  of 
simulation  fidelity. 

c.  Authors'  Conclusions:  The  use  of  aerolastic 
equations  simplified  by  rigid  coefficients  in  flight 
trainer  settings  provides  an  effective  training  basis 
for  subsequent  transfer  to  high  fidelity  simulation. 

Using  least  squares  approximations  to  the  flexible 
coefficients  in  the  aerodynamic  equations  during 
training  on  the  simulator  is  of  doubtful  value.  It 
seems  that  during  training,  the  least  squares  pilots 


At  all  stages  of  training  and  testing,  objective 
measuremens  of  error  and  deviation  were  made  both  of 
pilot  input  (manipulation  of  controls)  and  system  ouput 
(simulated  aircraft  f lightpath) ,  in  several  parameters. 
Subjective  evaluations  of  the  device  training  were  also 
made  by  subjects  and  controls  via  questionnaire. 

b.  Results: 

(1)  Key  Data: 

Comparisons  Between  Average  Performances  of 
Experimental  Group  (Rapid  Data)  Transfer  Trials 
with  Control  Group  (Flexible  Data)  on  Comparable 
Trials 


Phase 


Control 

Expermt. 

Ratio 

Prob 

Parameter 

Group(C) 

Group(E) 

E/C 

Altitude  Error 

208.00 

213.00 

1.02 

.50 

Heading  Error 

17.50 

16.80 

.96 

.45 

Climbing  Turn 

Fore/Aft 

Stick  Var. 

2.50 

3.10 

1.24 

.30 

Lateral 

Stick  Var. 

.37 

.37 

1.00 

.50 

Altitude  Error 

201.00 

180.00 

.89 

.45 

Heading  Error 

21.10 

22.20 

1.05 

.45 

Level  Turn 

Fore/Aft 

Stick  Var. 

2.60 

4.00 

1.54 

.20 

Lateral 

Stick  Var. 

.43 

.50 

1 .16 

.35 
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m.  Use  of  Instructional  Features 

(1)  Intensity:  intensive 

(2)  Features  used:  Number/Quality  of  Responses; 

others  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  This  study  continued  an  extensive 
program  of  research  on  varying  functional  fidelity  in  a 
jet  flight  simulator,  using  the  Universal  Digital 
Operational  Flight  Trainer  Tool,  a  high-speed  stored 
program  digital  computer  capable  of  being  programmed  to 
various  levels  of  fidelity.  The  purpose  of  this  study 
was  to  discover  if  reduced  levels  of  the  functional 
fidelity  in  the  dimension  of  flight  airframe 
coefficients  would  be  effective  in  training 
less-experienced  pilots  than  had  previously  undergone 
training  and  testing  on  the  device.  Previous 
experimentation  had  been  done  with  experienced  jet 
pilots;  the  subjects  in  this  experiment  were 
instrument  rated  reciprocating  engine  pilots  who  had 
never  flown  a  jet.  These  subjects  resembled  military 
student  pilots  although  they  did  have  instrument 
ratings  which  most  students  do  not. 

The  variable  of  interest  in  this  experiment  was  the 
flight  envelope  airframe  coefficient,  specific  to  any 
aircraft,  which  under  real  conditions  changes  with  both 
speed  and  altitude  (expressed  in  the  simulator  as  the 
"flexible"  condition) .  Two  reduced  fidelity 
experimental  conditions  were  effected:  (1)  "Rigid 
airframe",  where  the  coefficient  changed  with  speed  and 
not  with  altitude,  but  the  configuration  of  change 
closely  paralleled  the  changes  which  occur  in  the 
"flexible"  condition;  (2)  "least  squares 
approximation",  where  the  coefficient  changed  with 
speed  and  not  with  altitude,  but  was  a  straight  line  of 
constant  slope  not  paralleling  the  flexible  coefficient 
configuration. 

The  third,  "flexible"  condition  was  the  control 
condition  and  also  the  transfer  test  and  retention  test 
condition  for  all  groups  (executed  in  the  simulator). 

Three  groups  separately,  of  approximately  six  pilots 
each,  were  trained  on  the  three  conditions,  on  one 
maneuver:  a  360  degree  standard  rate  turn  with  a  2,000 

fpm  climb  in  the  first  half  and  constant  altitude 
during  the  second  half  of  the  turn.  All  subjects  and 
controls  were  then  tested  for  transfer  in  the 
"flexible",  high  fidelity  condition.  A  week  to  a  month 
later,  all  were  again  tested  for  retention. 
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data,  both  pilot  inputs  (manipulations  of 
controls)  and  system  outputs  (flight  path) 

(a)  average  absolute  deviations  from 
programmed  path 

(b)  average  algebraic  deviations 

(c)  mean  and  variance  of  airleron  and 
elevator  surface  motions,  fore/aft  stick, 
elevator  trim,  lateral  stick,  and  aileron 
trim 

(2)  Questionnaire  evaluations  of  device  training 
by  subjects  and  controls 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  non-parametr ic  Ratio  Tests; 
non-  parametric  Sign  Tests 

h.  Variables  being  manipulated: 

(1)  Training  Device:  Universal  Digital 
Operational  Flight  Trainer  Tool,  a  high-speed 
stored-program  digital  computer  with  two  simulator 
cockpits  and  an  instructor  station 

(a)  Rigid  airframe  coefficient  data  program 

(b)  Least-squares  airframe  coefficient  data 
program 

(c)  Flexible  airframe  coefficient  data 
(control  condition  and  all  transfer  and 
retention  test  condition) 

(2)  Fidelity  Levels: 

Rigid  Least  Sqs.  Flexible 
Physical  High  High  High 

Functional  High  Med-high  Very  High 

(3)  Type  of  Task/Skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  familiarization,  skill, 
transition 

j.  Trainee  Sophistication:  intermediate  to 
near-expert  (general  flying  experience  was  moderately 
high) 

k.  Incorporation  of  device  into  P.O.I.:  not 
applicable 

l.  User  Acceptance  of  Attitude: 

(1)  Instructors:  not  discussed  (experimenters  ran 
the  device) 

(2)  Students:  not  discussed,  except  insofar  as 
questionnaire  seemed  to  show  they  were  not  aware 
of  variations  in  fidelity  on  the  device 
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1.  Authors:  Ellis,  N.  C. ,  Lowes,  A.  L. ,  Matheny,  W.  G. ,  & 
Norman,  D.  A. 

2.  Title:  Pilot  Performance,  Transfer  of  Training,  and 
Degree  of  Simulation:  III.  Performance  of  Non-Jet 
Experienced  Pilots  Versus  Simulation  Fidelity 

3.  Source:  Technical  Report:  NAVTRADEVCEN  67-C-0034-1, 
August  1968 

4.  Topic  Keywords:  Flight  Training  ;  Flight  Simulation  ; 
Flexible  Coefficients  ;  Rigid  Coefficients  ; 

Least  Square  Approximation  ;  Transfer  of  Training  . 

5.  Short  Summary:  Varying  the  functional  fidelity  of  a 
simulator  during  flight  training  shows  that  some  conditions 
of  reduced  fidelity  are  as  effective  as  high  fidelity  in 
teaching  certain  maneuvers. 

6.  Device: 

Universal  Digital  Operational  Flight  Trainer  Tool,  a 
high-speed  stored-program  digital  computer  with  two 
simulator  cockpits  and  an  instructor  station. 

7.  Institutions: 

a.  Sponsor:  U.S.  Naval  Training  Device  Center, 
Orlando,  FL 

b.  Performing  Organization:  Life  Science,  Inc.  7305 
Grapevine  Highway,  Fort  Worth,  TX  76118 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  Approximately  6  instrument-rated 
pilots  without  jet  experience,  average  flight  time 
experience  approximately  500  hrs. 

(2)  Subjects:  approximately  6  same 

(3)  Controls:  approximately  6  same 

c.  Tests  or  Trials/timing:  10  training  trials;  10 
transfer  trials  (under  control  condition);  10 
post-training  retention  trials  approximately  1  week  to 
1  month  after  training  and  transfer  trials 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Objective  instrument-measured  performance 
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representative  of  real  flight  requirements. 
Measurements  of  error  were  made  in  the  pilots' 
performances  and  they  were  questioned  to  assess  their 
reactions  to  the  simulator's  characteristics. 

b.  Results: 

(1)  Key  Data:  Correlations  Between  Performance 
Measures  For  Longitudinal  and  Lateral  Parameters 

Performance 


Parameter 
(Abs.  Error) 

Pilot 

AE:p 

AE :  r 

AE  :a 

Altitude 

A 

.15 

— 

-.03 

B 

-.15 

-.05 

-.38 

C 

-.56 

.09 

-.51 

Rate  of  Climb 

A 

-.30 

— 

-.40 

B 

-.38 

.63 

-.74 

C 

— 

— 

— 

Pitch  Rate 

A 

.28 

— 

-.14 

B 

-.58 

.25 

-.87 

C 

-.15 

.64 

-.07 

Elevator 

A 

-.17 

— 

-.31 

Deflection 

B 

-.52 

.31 

-.68 

C 

-.09 

.53 

-.03 

(2)  Verbal  Description:  included  under 
CONCLUSIONS  below,  paragraph  c. 

c.  Authors'  Conclusions:  The  simulator  provided  a 
degree  of  simulation  adequate  for  investigation  of  many 
pilot-  vehicle  variables.  The  initial  group  of 
maneuvers  has  demonstrated  the  feasibility  of  using  a 
programmed  flight  path  as  a  basis  for  error  score 
computation. 


g.  Stat.ctical  Methods:  not  specified 

h.  Variables  being  manipulated: 

(1)  Training  Device:  Universal  Digital 
Operational  Flight  Trainer  Tool,  a  real-time 
single  jet  aircraft  simulator 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified 

(b)  Functional:  unspecified.  Functional 
fidelity  was  the  chief  variable  being  studied 

(3)  Type  of  task/skill  required:  aircraft 
piloting;  operations;  cognitive;  psychomotor; 
perceptual 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  not  applicable 

j.  Trainee  Sophistication:  expert 

k.  Incorporation  of  device  into  P.O.I.:  not 
.  applicable 

l.  User  Acceptance  or  Attitude:  not  specified 

m.  User  of  instructional  features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  This  paper  reports  on  a  phase  of  a 
developmental  study  evaluating  requirements  for 
aircraft  flight  simulation  through  the  use  of  the 
Universal  Digital  Operational  Flight  Trainer  Tool,  a 
real-time  single  engine  jet  aircraft  flight  simulator. 
This  multiple-programmable  device  was  capable  of 
producing  various  degrees  of  fidelity.  Topics  covered 
in  the  paper  were: 

Previous  UDOFTT  Studies 

Pilot  Vehicle  System  Variables 

Analysis  of  Piloting  Tasks 

Selection  of  Independent  Variables 

Variables  to  be  Controlled 

Performance  Measurement 

Pilot  Opinion 

Design  of  Maneuvers 

Experimental  Design 

Preliminary  Studies 

Technical  Requirements 

In  the  initial  pilot  performance  study,  three  highly 
experienced  jet  fighter  pilots  flew  a  series  of  6 
maneuvers  on  the  device,  considered  to  be 


1.  Authors:  Demaree,  Robert  G. ,  Norman,  Don  A.,  & 

Matheney,  William  G. 

2.  Title:  An  Experimental  Program  for  Relating  Transfer  of 
Training  to  Pilot  Performance  and  Degree  of  Simulation 

3.  Source:  NAVTRADEVCEN  Technical  Report  1388-1,  June  1965 

4.  Topic  Keywords:  Fidelity  ;  Flight  Simulation  ; 

Tranfer  of  Training  . 

5.  Short  Summary:  A  developmental  study  evaluated  a  number 
of  different  components  of  the  task  of  flying  a  jet  aircraft 
in  relation  to  flight  simulation  requirements. 

6.  Device: 

Universal  Digital  Operational  Flight  Trainer  Tool,  a 
real-time  single  engine  jet  aircraft  simulator 

7.  Institutions: 

a.  Sponsor:  U.S.  Naval  Training  Device  Center, 
Orlando,  FL  32813 

b.  Performing  Organization:  Life  Science,  Inc.  7305 
Grapevine  Highway,  Fort  Worth,  TX  76118 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  1 

b.  Description  of  Subjects:  3  highly  experienced  jet 
fighter  pilots 

c.  Tests  or  Trials/Timing:  6  maneuvers,  in-process 
trials 

d.  Number  of  different  types  of  measures:  2 

e.  Description  of  measurements  and  ratings: 

(1)  Errors  and  deviations: 

(a)  Mean  Squared  Error 

(b)  Average  Absolute  Error 

(c)  Maximum  deviation  of  a  parameter  from  its 
reference  value 

(d)  Real-time  graphic  recordings  of 
deviations 

(2)  Subject  evaluation  of  device 

f.  Experimental  Setting/Training  context:  Laboratory, 
hands-on 


(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  skill,  transition 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  not 
specified,  assumed  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  moderately  to  highly  favorable 
as  assessed  by  questionnaire 

(2)  Students:  same  as  instructors 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  assumed  intensive 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  This  study  evaluated  the 
effectiveness  of  the  Carrier  Air  Traffic  Control  Center 
( CATCC)  training  device  in  training  teams  to  safely  and 
efficiently  control  aircraft  recoveries  (landings)  and 
to  effectively  maintain  communications  necessary  to 
implement  this  control  function.  A  total  of  23  CATCC 
personnel  were  observed  in  the  trainer;  subsequently, 
15  of  these  were  observed  in  the  operational 
environment  while  performing  their  first  scheduled  Mode 
III  aircraft  recoveries.  These  recoveries  took  place 
two  months  after  the  final  training  period  and  were 
preceded  by  part-task  OJT  activities  in  preparation  for 
Mode  III  recoveries. 

Fourteen  personnel  from  Ship  One  were  observed  in  their 
1st  and  3rd  weeks  of  training;  9  of  these  on  shipboard 
(the  transfer  environment)  after  1  and  3  weeks  of 
device  training  and  the  2-month  OJT  interval.  Nine 
personnel  from  Ship  Two  were  observed  in  their  5th  week 
of  device  training,  and  6  of  these  on  shipboard  after  5 
weeks  of  device  training  and  the  2-month  OJT  interval. 
Device  effectiveness  was  to  be  assessed  relative  to  the 
hypothesized  superiority  of  Ship  Two  trainees  due  to 
their  more  lengthy  practice  on  the  simulator. 

A  large  number  of  measures  were  taken  to  evaluate 
swiftness,  accuracy,  and  safety  of  control  procedures 
conducted  by  the  trainees.  Data  were  analyzed 
separately  in  respect  to  team  performance,  subteam 
performance,  and  individual  performance. 


(1)  Key  Data:  Results  are  presented  in  a  43-page 
appendix  which  includes  19  graphs,  but  no  overall 
measures  which  can  be  extracted  or  summarized 
here. 

(2)  Verbal  Description:  Team,  subteam,  and 
individual .performance  clearly  improved  as  a 
result  of  device  utilization.  Students  showed 
evidence  of  learning  to  contend  with  recovery 
contingencies  and  emergencies  in  the  training 
device.  The  objective  performance  data  indicate 
that  device-trained  students  do  perform  acceptably 
in  the  operational  setting  after  completion  of 
training.  Both  students  and  instructors  rated  the 
trainer  as  moderately  to  highly  effective.  Both 
students  and  instructors  rate  the  trainer  as 
moderately  to  highly  realistic  in  both  dimensions 
of  task  environment  and  task  performance  (physical 
and  functional) . 

c.  Authors'  Conclusions:  The  data  indicate  that  each 
team  and  subteam  composition  tends  to  be  a  somewhat 
unique  entity  which  benefits  from  training  as  an 
entity. 

The  differences  in  measured  and  observed  performances 
between  teams,  subteams  and  individuals  support  the 
transfer  of  training  hypothesis  in  that  personnel  with 
more  training  performed  better.  The  evidence  is  not 
conclusive,  however,  because  the  study  could  not  be 
structured  so  as  to  control  or  positively  account  for 
possible  confounding  variables. 


1.  Authors:  Goebel,  Ronald  A.,  Baum,  David  R. ,  & 

Hagin,  William  V. 

2.  Title:  Using  a  Ground  Trainer  in  a  Job  Sample  Approach 
to  Predicting  Pilot  Performance 

3.  Source:  Technical  Report:  AFHRL-TR-71-50 ,  1971 

4.  Topic  Keywords:  Job  sample  ;  Flight  simulation  ; 
Screening  . 

5.  Short  Summary:  Evaluation  of  the  GAT-1  light  plane 
flight  simulator  as  a  predictor  of  student  pilot  performance 
in  a  T-41  light  plane  and  a  jet  trainer  increases  confidence 
in  the  ability  of  the  GAT-1  to  predict  flying  performance 
via  the  "job  sampling"  method. 

6.  Devices: 

a.  T-37  twin  engine  jet  trainer 

b.  T-41  light  airplane 

9  c.  Specially  instrumented  Link  B-Model  GAT-1  trainers, 

slightly  modified  for  recording  purposes 

7.  Institutions: 

a.  Sponsor:  Air  Force  Human  Resources  Laboratory  , 
Brooks  Air  Force  Base,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
AFHRL,  Air  Force  Systems  Command,  Williams  Air  Force 
Base,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2,  although  there  were 
ill-defined  sub-groups 

b.  Description  of  Groups: 

(1)  Subjects:  Approximately  87  undergraduate  jet 
pilot  trainees 

(2)  Controls:  49  same 

c.  Tests  or  Trials/Timing: 

(1)  In-process  performance  evaluation  (simulator) 

(2)  T-41  checkride 

(3)  T-37  checkrides:  midphase  and  final 
a.  Number  of  Different  Types  of  Measures:  9 
e.  Description  of  Measurements  and  Ratings: 
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(1)  Subjective  evaluation  of  simulator  performance 
by  instructors 

(2)  T-41  numerical  score  by  maneuver 

(3)  T-41  overall  nominal  4-category  scale  score 

(4)  T-41  adjusted  overall  grade 

(5)  T-37  midphase  overall  nominal  (4-category) 

(6)  T-37  midphase  adjusted  overall  nominal 
( 4-category) 

(7)  T-37  final  contact  flight  grade 

(8)  T-4  instrument  procedures  trainer  (final 
grade) 

(9)  T-37  final  instrument  grades 


f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 


g.  Statistical  Methods:  Pearson  correlations;  others 
not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  T-37  twin  engine  jet  trainer 

(b)  T-41  light  airplane 

(c)  Specially  instrumented  Link  B-Model  GAT-1 
trainer,  slightly  modified  for  recording 
purposes 

(2)  Fidelity  Levels: 

(a)  Physical:  not  specified,  presumably 
medium 

(b)  Functional:  not  specified,  presumably 
medium 

(3)  Type  of  Task/Skill  Required:  aircraft 
piloting;  operations,  psychomotor,  perceptual; 
cognitive,  part-task 

(4)  Task  Difficulty:  medium  and  high 

i.  Stage  of  Training:  introduction,  transition, 
familiarization,  skill 

j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable  (screening,  not  training  application) 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  favorable  as  judged  by 
experimenters 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified,  presumed  intensive 

(2)  Features  used:  not  specified 


Abstract : 


a.  Study  Synopsis:  Challenges  to  the  assumption  that 
discrete  abilities  measured  by  aptitude  tests  are 
stable  and  reliable  predictors  of  the  success  of  jet 
flight  trainees  have  pointed  to  "job  sampling"  as  a 
possible  alternative  method  of  screening  candidates. 

In  the  context  of  this  study,  "job  sampling"  consists 
of  "obtaining  work  samples  during  early  training  or 
simulating  work  situations  prior  to  training  and 
deriving  measures  of  performance  from  either  to  employ 
as  predictors  of  future  job  success."  At  the  time  of 
the  study,  the  Air  Force  was  using  the  T-41  light  plane 
as  a  screening  device  for  student  jet  pilot  candidates. 
There  was  evidence  that  a  simulator  might  also  be  used 
as  a  job  sampling  device. 

The  present  experiment  was  designed  to  evaluate  both 
the  Link  GAT-1  trainer  and  the  T-41  light  aircraft  as 
predictors  of  student  pilot  performance  in  the  T-37 
twin-  engine  jet  trainer.  Prior  to  T-37  training, 
trainees  in  the  experiment  received  sorties  in  the 
GAT-1  and  conventional  T-41  sorties  on  either  of  two 
schedules:  (1)  6  1-hour  sorties  prior  to  T-41  phase  of 

training;  (2)  3  sorties  subsequent  to  T-41  graduation. 
A  control  group  received  no  GAT-1  sorties. 

At  the  time  of  this  report,  only  one  predictor  variable 
was  used,  a  subjective  evaluation  of  trainees  by 
instructors  based  on  the  subjects'  performance  in  the 
GAT-1  (several  quantitative  measures  were  taken  but  the 
analysis  of  data  had  not  been  completed  at  the  time  of 
the  report) . 

Criterion  variables  were  (1)  for  the  T-41,  numerical 
and  overall  nominal  grades,  and  adjusted  overall  grade; 
(2)  for  the  T-37,  midphase  overall  nominal  and  adjusted 
overall  nominal  grades,  and  final  contact  grade,  final 
procedures  trainer  (T-4)  grade,  and  final  instrument 
grade . 

Correlations  were  made  between  the  predictor  variable 
and  the  criterion  variables,  from  the  GAT-1  to  the 
T-41,  from  the  T-41  to  the  T-37,  and  from  the  GAT-1  to 
the  T-  37. 
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b.  Results: 

(1)  Key  Data:  (note:  numbers  in  parentheses  are 
the  numbers  of  subjects  in  each  correlation  (N)  )  . 

(a)  Correclation  Coefficients  Between  GAT-1 
Instructor  Evaluations  and  Final  T-41 
Adjusted  Grade 

Average  all  classes:  df=104;  r=+0.50, 

significant  at  .01  level 

(b)  Correlation  Coefficients  Between  T-41 
Adjusted  Grades  and  T-37  Adjusted  Creiteria 

_ T-37  Final _ 

Class  Midphase  Trainer  Instrument  Contact 

72-04 

(control)  .24  (49)  .35*(49)  . 38** (49)  .29*  {  49 ) 

72-05  -.09  (32)  .52**  (31)  -.10  (31)  -.04  (31) 

Avg  (experi¬ 
mental 

class)  .12  (87df )  . 38** ( 84df ) . 16  (84df)  .19  (84df) 

Note:  Sample  size  indicated  in  parentheses.  * 

Significant  at  .05  level.  **  Significant  at  .01 
level . 

(c-)  Correlation  Coefficients  Between  GAT-1  IP 
Grades  and  T-37  Adjusted  Criteria 

_ T-37  Final _ 

Class  Midphase  Trainer  Instrument  Contact 

72-04  (control)  -  -  - 

72.05  .14  (32)  .45** (31)  .27  (31)  .31  (31) 

Avg.  . 23* (87df )  .29**(84df)  .16(84df)  .30**(84df) 

*  Significant  at  .05  level.  **  Significant  at  .01 
level . 

(2)  Verbal  Description:  Correlation  is  high 
between  the  GAT-1  instructor  evaluations  and  final 
T-41  adjusted  grades  (table  a).  Results  partially 
shown  in  table  b  have  significant  correlations 
between  T-41  grade  and  T-37  grades  in  5  out  of  12 
categories.  Only  1  out  of  12  of  the  correlations 
between  the  GAT-1  and  the  T-37  was  significant. 
However,  in  comparing  average  correlations  between 
tables  (b)  and  (c) ,  no  statistically  significant 
differences  were  obtained. 


c.  Authors'  Conclusions:  It  is  likely  that  measures 
from  the  light-plane  ground  trainer  can  be  used  to 
predict  subsequent  light  aircraft  performance.  Also, 
there  is  often  a  significant  relationship  between 
light-plane  flying  grades  and  subsequent  jet  aircraft 
performance.  The  lack  of  significant  differences 
between  the  average  correlations  of  T-41-to-T-37  and 
GAT-l-to-T-37  suggests  a  tentative  interpretation  that 
a  subjective  measure  of  GAT-1  performance  can 
potentially  result  in  prediction  equivalent  to  that 
obtainable  with  a  like  measure  of  light-plane 
performance.  The  present  study  has  resulted  in  enough 
increased  confidence  in  the  ability  of  the  GAT-  1 
ground  trainer  to  "sample  the  job"  that  research  may 
not  be  addressed  to  the  question  of  screening  with  a 
ground  trainer  in  the  absence  of  light-plane  training. 


1.  Authors:  Goett,  James  M.  ,  Post,  Theodore  J.,  & 

Miller,  Gary  G. 

2.  Title:  6883  Maintenance  Training  Simulator  Development 
Utilizing  Imagery  Techniques. 

3.  Source:  Air  Force  Human  Resources  Laboratory  TR-80-3, 
May  1980 . 

4.  Topic  Keywords  :  Imagery  ;  Maintenance  Training  ; 
Simulation  ;  Low  Fidelity  ;  Training  Effectiveness  . 

5.  Short  Summary:  This  study  used  a  low  fidelity,  imagery- 
based  simulation  technique  in  training  maintenance 
procedures.  The  study  produced  mixed  and  inconclusive 
results,  with  the  experimental  group  excelling  on  the  most 
difficult  task  and  the  control  group  excelling  on  the 
simplest  task. 

6.  Devices: 

a.  6883  Test  Station. 

b.  Low  Fidelity  Graphics  -  Line  drawings  with  acetate 
overlays  and  marking  styli  for  use  with  photographs  of 
equipment. 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  BioTechnology ,  Inc.,  3027 
Rosemary  Lane,  Falls  Church,  VA  22042 

8.  Type  of  Article:  Experiment. 

a.  Design  Structure  Elements: 

(1)  Number  of  Groups:  2 

(2)  Testing:  Post-Tests  only 

(3)  Statistical  Methods:  ANOVA 

b.  Subjects:  26  Air  Force  students  in  maintenance 
training 

c.  Controls:  27  same 

d.  Variables  Being  Manipulated: 

(1)  Type  of  Training  Device:  Line  drawings  with 
acetate  overlays  and  marking  styli  for  use  with 
photographs  of  equipment 

(2)  Fidelity  Level:  Low 


(3)  Type  of  Task:  Maintenance  Procedures 

(4)  Task  Difficulty:  Low 

1 5 )  Trainee  Sophistication:  Unspecified; 
presumed  moderate 

(6)  Stage  of  Training:  Unspecified 

(7)  Training  Context:  School 

(8)  User  Attitude:  Indifferent,  as  indicated  by 
questionnaire 

e.  Dependent  Variables  (effects  of  manipulation) 

(1)  Performance  Measures:  Written  test  scored  by 
percentage  of  correct  steps  specified  in  each  of 
four  tasks 

(2)  Supervisor  Ratings:  None  applicable 
Abstract : 

a.  This  exploratory  study  evaluated  a  low  fidelity, 
imagery-based  simulation  technique  to  determine  its 
effectiveness  in  training  certain  avionics  maintenance 
tasks.  Subjects  and  controls  were  taught  maintenance 
procedures  for  the  6883  Test  Station  employing  (1)  for 
experimental  subjects,  imagery-generating  materials 
with  a  photographic  mock-up  of  test  station,  (2)  for 
controls,  conventional  written  lists  of  instructions 
with  photographic  mock-up  only.  Testing  was  by  written 
answers,  referring  to  photographic  mock-up  of  the  6883 
station. 

Four  different  tasks  were  taught;  after  training  for 
each  task,  a  test  for  that  task  was  given,  then  the 
next  task  was  taught.  At  the  conclusion  of  learning 
and  being  tested  on  the  four  tasks,  students  took  a 
repeated,  "retention"  test  for  Task  1  (a  period  of 
about  2  hours  had  elapsed  since  the  first  test  for  Task 
1  had  been  given).  (See  Verbal  Description  of 
results) . 

The  fact  that  the  experimental  group's  performance  on 
the  "retention"  test  of  Task  1  fell  to  the  level  of  the 
control  group's,  while  the  control  group  showed  no  loss 
of  learning,  could  be  attributed  to  one  or  either  of 
two  possibilities:  (1)  the  facilitation  of  the  control 
group's  learning  through  the  training  and  testing  on 
Tasks  2,  3,  and  4;  (2)  the  interference  with  the 

experimental  group's  learning  during  the  same  series. 
(See  Authors'  Conclusions). 


b. 


Results : 

(1)  Key  Data:  Mean  Percentage  Correct  for  Groups 
on  the  Learning  and  Retention  Tests  of  Task  1  (the 
most  complex  of  the  4  tasks)  : 


Group 

Experimental  (N=26) 
Control  ( N=27 ) 


Learning 
MEAN  (Std) 
64.8  (11.2) 
52.6  (21.0) 


Retention 
MEAN  (Std) 
51.7  (14.8) 
51.0  (18.5) 


(2)  Verbal  Description:  On  tests,  groups  using 
the  imagery  training  materials  (experimental) 
scored  significantly  better  than  groups  using  the 
conventional  materials  (control)  on  one  out  of 
four  tasks  (the  most  lengthy  and  complex) ;  on  a 
retention  test  for  that  same  task,  scores  were  not 
significantly  different  due  to  a  decrement  in 
experimental  group  performance.  The  control  group 
significantly  outscored  the  experimental  group  on 
one  of  the  four  tasks  (the  shortest  and  simplest) . 


c.  Authors'  Conclusions:  In  general,  these  evaluation 
results  were  mixed  with  respect  to  the  effectiveness  of 
imagery  versus  conventional  training  materials.  The 
results  suggest  that  the  imagery  training  technique  may 
be  more  appropriate  for  training  more  complex 
procedural  tasks  and  tasks  with  higher  spatial  and 
psychomotor  components.  The  unexpected  success  of  the 
conventional  training  materials  suggested  the  value  of 
the  equipment  mock-up  (photograph  without  other  graphic 
aids)  for  procedural  training. 


1.  Authors:  Goldin,  Sarah  E.  , 


&  Thorndyke,  Perry  W. 


2.  Title:  Simulating  Navigation  for  Spatial  Knowledge 
Acquisition. 

3.  Source:  Human  Factors  ,  1982,  24(4),  457-471 

4.  Topic  Keywords:  Navigation  Simulation  ; 

Spatial  Knowledge  Acquisition  . 

5.  Short  Summary:  This  study  compared  different  methods  of 
information  presentation  in  testing  location  knowledge 
acquisition.  The  different  sources  of  environmental 
information  were  an  actual  bus  tour  through  an  unfamiliar 
city  area,  and  a  filmed  version  of  the  same  route.  The 
results  revealed  that  filmed  navigation  can  be  used  as  an 
effective  substitute  for  actual  environmental  experiences 
under  some  circumstances,  especially  for  familiarization. 

6.  Devices: 


a. 

Bus  tour 

b. 

Moving  picture  film 

c. 

Maps 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  , 
Alexandria,  VA  22333. 

b.  Performing  Organizations:  The  Rand  Corporation, 
Santa  Monica,  CA,  and  Perceptronics,  Inc. ,  Menlo  Park, 
CA. 

8.  Experiment: 

a.  Number  of  groups:  6 

b.  Description  of  groups: 

( 1 ) — ( 4 )  Subjects:  UCLA  undergraduates,  randomly 
assigned  from  pool  of  94  (29  males  and  65  females) 
(5)-(6)  Controls:  Same 

c.  Tests  or  Trials/Timing:  Post-tests  (battery); 
also  aptitude  tests  for  general  ability  (not  measuring 
dependent  variable) 

d.  Number  of  different  types  of  measures:  6 
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e.  Description  of  measurements  and  ratings: 

(1)  Descriminability  in  location  recognition 
(right/wrong) 

(2)  Location  sequencing  (right/wrong) 

(3)  Estimated  distances  on  route  (continuum) 

(4)  Angular  error  of  orientation  (continuum) 

(5)  Angular  error  of  landmark  placement  on  map 
(continuum) 

(6)  Estimated  Euclidean  distances  on  map 
(continuum) 

f.  Experimental  setting/training  context: 

Laboratory —  classroom 

g.  Statistical  methods:  ANOVA;  other  (if  any)  not 
specified 

h.  Variables  being  manipulated: 

(1)  Training  Devices 

(a)  Bus  Tour 

i.  with  map  supplementation 

ii.  with  narrative  supplementation 

iii.  no  supplementation  (control) 

(b)  Film 

i.  with  map  supplementation 

ii.  with  narrative  supplementation 

iii.  no  supplementation  (control) 

(2)  Fidelity  Levels  (of  filmed  simulation) 

(a)  Physical:  medium 

(b)  Functional:  none  (passive  learning) 

(3)  Type  of  task/skill  required:  navigation; 
cognitive;  perceptual 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  training:  experimental,  familiarization 

j.  Trainee  sophistication:  novice 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  of  attitude:  not  discussed 

m.  User  of  instructional  features:  Intensity  - 
incomplete 


9.  Abstract: 


a.  Study  Synopsis:  To  compare  actual  and  simulated 
navigation  as  sources  of  evironmental  information,  an 
actual  bus  tour  of  an  unfamiliar  city  environment  was 
compared  with  a  film  of  the  same  route  shot  from  inside 
an  automobile.  Subjects  and  controls  were  divided  into 
6  groups  randomly  from  a  pool  of  94  UCLA  undergraduates 
(29  males  and  65  females).  Three  groups  of  subjects 
rode  the  bus  tour:  of  these,  one  group  was  given  a 
supplementary  map;  another  received  an  en-route 
narrative  from  an  experimenter;  the  third,  control 
group,  received  no  supplementary  aid.  Three  groups 
viewed  the  film,  and  received,  respectively,  the  same 
supplementation. 

Following  the  actual  and  filmed  "tours",  all  groups 
took  a  battery  of  6  tests  designed  to  elicit  their 
knowledge  of  landmarks,  of  sequence  of  landmarks  on  the 
route,  a  distance  between  points  on  route  of  direction 
of  various  landmarks  from  various  positions  on  the 
route,  and  of  configural  (map)  relationships  between 
parts  of  the  route.  They  were  also  given  tests  of 
general  visual-  spatial  aptitude,  to  examine  the 
possibilities  of  between-gr oup  differences  in  ability. 


b.  Results: 

(1)  Key  Data:  Summary  of  comparisons  between  Film 
and  Tour  Groups 


Task  Type 

Landmark  Knowledge: 
Location  recognition 

Procedural  Knowledge: 
Location  sequencing 

Route  distance 
estimation 

Orientation  judgment 


Result 

Film  groups  more  accurate 
than  tour  groups 


Film  groups  more  accurate 
than  tour  groups 

No  difference  between 
film  _tour 

Tour  groups  more  accurate 
than  film  groups 


Survey  Knowledge  (configural): 

Landmark  placement  No  difference  between 

film  and  tour  groups 


Euclidean  distance 
estimation 


No  difference  between 
film  and  tour  groups 
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(2)  Verbal  Description:  The  live  tour  appeared 
superior  to  the  filmed  tour  in  only  one  of  the 
measured  variables — "orientation  judgement",  which 
tested  the  ability  of  subjects,  after  envisioning 
themselves  at  various  positions  along  the  route, 
to  indicate  what  direction  certain  landmarks  lay. 
However,  neither  group  performed  very  accurately 
on  this  task. 

The  influence  of  supplementary  information  on 
subjects'  acquired  knowledge  was  more  complex  than 
had  been  expected.  First,  only  the  film  groups 
were  influenced  by  supplementary  information. 
Narrative  supplements  degraded  performance  when 
they  affected  it  at  all.  Map  supplements,  on  the 
other  hand,  sometimes  enhanced  performance  and 
sometimes  degraded  it. 

c.  Authors'  Conclusions:  Perceptual  knowledge  of 
locations  in  the  environment  can  apparently  be  acquired 
at  least  as  well  from  a  film  as  from  actual  navigation. 
Live-tour  experience  appears  superior  to  simulated 
navigation  for  the  acquisition  of  only  one  compenent  of 
procedural  knowledge:  the  angles  of  turns  along  the 
route.  However,  neither  group  performed  very 
accurately  on  the  orientation  task.  At  least  a  quarter 
of  the  subjects  in  every  group  were  "disoriented" 
according  to  the  criterion  (90  degrees  off  correct 
bearing) .  This  low  level  of  performance  probably 
resulted  from  3  factors:  the  size  and  complexity  of 
the  environment,  the  limited  exposure  euv,h  subject  had 
to  the  environment,  and  the  passive  nature  of  the 
experience. 

This  study  demonstrates  that,  under  some  conditions, 
environmental  knowledge  acquired  through  simulated 
navigation  can  be  as  accurate  and  as  complete  as 
knowledge  acquired  through  live  navigational 
experience.  In  particular,  when  the  goal  is  to  convey 
either  visual  detail  or  configural  relationships,  a 
film  may  provide  at  least  as  much  information  as  a 
single  live  tour. 


b.  Results:  Mean  scores  on  transfer  task  by  training 
group. 


Hot 

Cold 

Repro 

Time  to  Train 

(min. ) 

Mean 

47.3 

45.8 

46.7 

SD 

10.6 

16.7 

12.2 

Proficiency 

Mean 

90.6 

89.4 

89.3 

SD 

1.4 

1.3 

1.8 

Retest  1 

Mean 

80.2 

79.8 

78.5 

SD 

4.2 

4.0 

3.0 

Retest  2 

Mean 

83.5 

84.2 

85.3 

SD 

2.8 

3.2 

5.0 

Trials  to  Retrain 

Mean 

2.3 

2.3 

1.8 

SD 

0.5 

0.6 

1.0 

Time  to  Retrain  (min.) 

Mean 

17.3 

17.0 

13.4 

SD 

3.0 

5.8 

3.9 

•  c.  Author's  Conclusions:  The  fidelity  of  training 

devices  used  to  train  men  to  perform  procedural  tasks 
can  be  very  low  without  adverse  effect  on  training 
time,  level  of  proficiency,  retention,  or  time  to 
retrain.  This  is  true  whether  the  trai  ing  is 
individually  or  group  administered.  Brief  practice  on 
the  high  fidelity  device  facilitates  the  performance  of 
groups  trained  on  the  low  fidelity  panel.  The  low 
fidelity  panel,  in  conjunction  with  a  list  of  the 
correct  actions,  can  be  used  to  effectively  reinstate  a 
high  level  of  performance  after  a  passage  of  time, 
regardless  of  the  panel  used  for  original  training. 

A  careful  review  of  tasks  to  be  taught  should  precede 
selection  of  training  devices.  Low  fidelity  devices 
may  be  used  to  considerable  advantage  both  for  economy 
in  training  and  for  effectiveness  of  training, 
remembering,  and  retraining. 

It  should  be  noted  that  the  author's  general 
conclusions  are  inferred  from  a  fixed  procedural  task 
at  a  fairly  low  level  of  complexity  (there  is  no 
branching  flow  in  the  task  sequence).  This  is 
recognized  by  the  author's  statement:  "the  implication 
of  this  study  is  not  that  simple  trainers  may 
substitute  for  complex  simulators,  but  rather  that,  for 
procedural  tasks  and  for  early  stages  of  certain  types 
of  training,  devices  other  than  procedural  trainers  are 
uneconomical  and  unnecessary." 
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h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Hot  Panel  (High  Fidelity) :  physical 
duplicate  of  the  tactical  panel  in  which  all 
lights,  meters,  intercom  and  other  indicators 
functioned 

(b)  Cold  Panel  (Medium  Fidelity) :  same  as 
above  except  with  no  electric  power 

(c)  Reproduced  Panel  (Low  Fidelity): 
full-size  artist’s  reproduction  of  Hot  Panel 

(2)  Fidelity  Levels:  High,  Medium,  Low 

(3)  Type  of  Task/Skill  Required:  92-step 
procedural  task 

(4)  Task  Difficulty:  Moderate  (simple  but  long) 

i.  Stage  of  Training:  Advanced  Individual  Training 

j.  Trainee  Sophistication:  Low 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed  § 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  Among  120  subjects  divided  into  10 
groups  learning  92-step  procedural  task  on  devices  of 
differing  simulator  fidelity,  the  results  showed  no 
significant  differences  in  training  time,  initial 
performance  level,  amount  remembered  after  4  and  6 
weeks,  or  retraining  time,  between  groups  trained  on 
high  and  those  trained  on  low  fidelity  devices. 
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1.  Author:  Grimsley,  Douglas  L. 

2.  Title:  Acquisition,  Retention,  and  Retraining:  Group 
Studies  on  Using  Low  Fidelity  Training  Devices 

3.  Source:  HumRRO  Technical  Report  69-4,  1969 

4.  Topic  Keywords:  Procedural  Task  ;  Fidelity  ; 

Low  Fidelity  ;  Retention  ;  Retraining  ;  Group  Training  . 

5.  Short  Summary:  Groups  trained  on  devices  of  differing 
fidelity  show  no  significant  differences  in  acquisition  and 
retention  of  training  in  procedural  tasks.  This  data 
indicates  that  low  fidelity  simulators  can  be  used  for  such 
training  without  adverse  effects  on  training  time, 
retention,  or  retraining  time. 

6.  Devices:  Three  simulated  display  and  control  panels  of 
differing  fidelity:  Hot  panel  (high  fidelity),  cold  panel 
(medium  fidelity),  reproduced  panel  (low  fidelity). 


7.  Institutions: 


a.  Sponsor:  Chief  of  Research  and  Development, 
Department  of  the  Army 

b.  Performing  Organization: 

George  Washington  University  Human  Resources  Research 
Office  Div.  3,  Presidio  of  Monterey,  CA  93940 

8.  Experiment: 


a.  Number  of  Groups:  Ten  Grouf  Post  Test 

b.  Description  of  Groups: 

(1)  Subjects:  120  Army  trainees,  excluding  Mental 
Category  IV,  sex  unspecified 

(2)  Controls:  none 

c.  Tests  or  Trials/Timing:  Post-tests  over  a  six  week 
period 


a.  Number  of  different  types  of  measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Time  to  train 

(2)  Proficiency  scores 

(3)  Time  to  retrain 


f.  Experimental  Setting/Training  Context:  School 

g.  Statistical  Methods:  ANOVA 

[  PREVIOUS  PA 
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(2)  Verbal  Description:  No  measurable  differences 
in  training  time  to  learn  the  procedural  task, 
initial  performance  level,  amount  remembered  after 
4  and  6  weeks,  or  retraining  time,  between 
individuals  trained  on  high,  and  those  trained  on 
lower  fidelity  devices. 

c.  Author's  Conclusions:  The  fidelity  of  training 
devices  used  to  train  individuals  on  procedural  tasks 
can  be  very  low  with  no  adverse  effect  on  training 
time,  level  of  proficiency,  retention,  or  time  to 
retrain.  Training  device  selection  should  be  based  on 
a  careful  review  of  the  tasks  to  be  taught. 
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b.  Results: 

(1)  Key  Data:  Mean  scores  on  independent 
variables  for  experimental  groups.  Treatment 
Groups:  H/H=  Trained  on  the  hot  panel  and  tested 

on  the  hot  panel;  C/H=  Trained  on  the  cold  panel 
and  tested  on  the  hot  panel;  C/C=  Trained  on  the 
cold  panel  and  tested  on  the  cold  panel;  R/H= 
Trained  on  the  repro  panel  and  tested  on  the  hot 
panel;  R/R=  Trained  on  the  repro  panel  and  tested 
on  the  repro  panel. 


Test 

H/H 

C/H 

C/C 

R/H 

R/R 

AFQT  Score 
Mean 

78.1 

78.8 

58.4 

79.2 

70.5 

SD 

GT  Score 

22.3 

20.2 

20.0 

10.3 

23.2 

Mean 

122.0 

124.0 

106.0 

126.0 

116.0 

SD 

17.7 

16.9 

21.7 

11.9 

17.9 

Time  to  Train 

(minutes) 

Mean 

114.0 

113.3 

118.3 

97.8 

132.3 

SD 

21.9 

30.1 

30.0 

30.5 

37.2 

Proficiency 

Score 

Mean 

90.9 

89.2 

90.1 

88.3 

89.5 

SD 

1.0 

3.1 

1.6 

3.4 

3  .6 

Retest  1 
Score 

Mean 

75.7 

75.0 

75.4 

75.1 

71.7 

SD 

Retest  1 
Score 

5.2 

4.3 

6  .1 

8.0 

8.3 

Mean 

82.9 

83.3 

83.3 

83.6 

83.3 

SD 

Trials  to 
Retrain 

4.6 

4.8 

6.5 

5.0 

5.5 

Mean 

2.5 

2.5 

2  .3 

2.2 

2.5 

SD 

1.0 

0.8 

0.4 

0.7 

1.0 

Time  to  Retrain 

(minutes) 

Mean 

20.7 

19.9 

19.0 

17.8 

21.1 

SD 

10.3 

6.9 

4.0 

8.3 

10.4 

Analyses  of  variance  for  these  groups  showed  that 
difference  were  not  significant. 
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The  task  to  be  learned  required  performing  92  simple 
procedural  tasks  (operation  of  switches,  plugging  in 
headsets,  making  brief  verbal  announcements,  etc.)  in 
proper  sequence.  Five  groups  of  12  subjects  each  were 
given  training  on  three  different  devices:  one  group 
on  the  "Hot  Panel",  two  groups  on  the  "Cold  Panel",  and 
two  groups  on  the  "Repro  Panel"  (High,  Medium,  and  Low 
Fidelity) .  Training  was  closely  monitored  and  assisted 
by  instructors  on  all  panels,  through  unsystematic 
reinforcement  and  cueing.  For  events  where  only  the 
powered  "Hot  Panel"  gave  automatic  feedback  to  subject, 
the  instructor  provided  verbal  feedback.  Subjects  on 
"Cold"  and  "Repro"  panels  were  required  to  speak  and 
mime  certain  actions  which  could  not  be  performed  on 
their  panels.  Training  to  error-free  completion  of 
sequence  required  a  maximum  3  hours  per  individual. 

Immediately  following  training,  subjects  were  tested 
and  scored  by  alternate  instructors,  some  on  "Hot", 
some  on  "Cold",  some  on  "Repro"  panels  (instructors 
merely  counted  number  of  correct  steps  completed) . 

Then  after  4  weeks,  and  again  after  additional  2  week 
"layoffs",  subjects  were  retested,  at  this  point  all  on 
*5|P)  9  the  "Hot  Panel"  —  see  table  below  duplicated  from 

report's  "Figure  2".  Following  the  tests  they  were 
retrained  to  correct  errors  and  their  retraining 
performance  was  recorded.  There  was  no  significant 
difference  between  the  various  trainees'  performances 
on  any  tests,  nor  were  there  significant  differences  in 
their  time  and  accuracy  in  relearning  to  correct  tested 
errors. 

All  subjects  retained  material  equally  well  for  6  weeks 
regardless  of  fidelity  of  device.  Furthermore,  groups 
remembered  equally  well  even  when  they  had  not  even 
seen  the  high  fidelity  device  at  any  point  in  their 
training. 
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d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings: 

(1)  Performance  Measures:  Three  proficiency  tests 
over  period  of  six  weeks,  scored  by  number  of 
steps  performed  correctly,  without  regard  to  time 

(2)  Supervisor  Ratings:  none 

f.  Experimental  Setting/Training  Context:  School 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  High  Fidelity  "Hot  Panel"  -  physical 
duplication  of  tactical  panel  in  which 
lights,  meters,  intercom,  and  other 
indicators  worked 

(b)  "Cold  Panel"  -  Medium  Fidelity  - 
identical  to  (a)  but  without  power 

(c)  "Repro  Panel"  -  Low  Fidelity  -  full-sized 
color  artist's  representation  of  (a) 

(2)  Fidelity  Levels:  High,  Medium,  Low 

(3)  Type  of  Task/Skill  Required:  92-step 
procedural  task,  stresses  learning  of  correct 
sequence  and  completion  of  all  steps 

(4)  Task  Difficulty:  Low  to  moderate 

i.  Stage  of  Training:  Advanced  Individual  Training 

j.  Trainee  Sophistication:  Low 

k.  Incorporation  of  Device  into  P.O.I.: 

l.  User  Acceptance  or  Attitude:  Not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  Low- to-moderate 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  Results  of  experiments  studying 
the  effect  of  fidelity  of  training  devices  on  retention 
of  learning  were  scarce  and  ambiguous  at  time  of  report 
(1969).  This  study  (STRANGER)  extends  work  of  a 
previous  study  (RINGER)  from  examining  acquisition  only 
to  examining  retention. 
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1.  Author:  Grimsley,  Douglas 

2.  Title:  Acquisition,  Retention,  and  Retraining:  Effects 
of  High  and  Low  Fidelity  in  Training  Devices 

3.  Source:  George  Washington  University  HumRRO  Technical 
Report  69-1,  1969,  p.  24 

4.  Topic  Keywords:  Retention  ;  Low  Fidelity  ;  Fidelity  ; 
Procedural  Tasks  ;  Retraining  ;  Medium  fidelity  ; 

High  fidelity  . 

5.  Short  Summary:  Subjects  were  trained  to  do  a  procedural 
task  on  three  devices  of  differing  fidelity.  No  significant 
differences  were  found  between  subjects  with  regard  to 
acquisition,  retention,  or  retraining  of  ability  to  perform 
this  task.  The  resulting  implication  is  that  procedural 
tasks  can  be  trained  on  low  fidelity  simulators  without 
adverse  effects. 

6.  Devices:  See  Section  8h(l)  below 

a.  High  Fidelity  "Hot  Panel"  -  physical  duplication  of 
tactical  panel  in  which  lights,  meters,  intercom,  and 
other  indicators  worked 

b.  "Cold  Panel"  -  Medium  Fidelity  -  identical  to  "a" 
but  without  power 

c.  "Repro  Panel"  -  Low  Fidelity  -  full-sized  color 
artist's  representation  of  "a" 

7.  Institutions: 

a.  Sponsor:  Office,  Chief  of  Research  and 
Development,  Department  of  the  Army 

b.  Performing  Organization:  Human  Resources  Research 
Office  Div.  No.  3,  Presidio  of  Monterey,  CA  93940; 
George  Washington  University 

8.  Experiment: 

a.  Number  of  Groups:  Three  groups,  three  post-tests 

b.  Description  of  Groups: 

(1)  Subjects:  60  trainees  in  Advanced  Individual 
Training,  all  with  minimum  score  of  30  on  Armed 
Forces  Qualification  Test. 

(2)  Controls:  None  (subjects  all  received  varying 
degrees  of  simulator  training) 

c.  Tests  or  Trials/Timing:  post-tests 
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showed  the  simulator  l/18th  as  expensive  as 
the  aircraft  per  contact  attempt. 

(b)  Study  II:  actual-aircraft  checkride 
performance  of  the  simulator-trained  group 
was  not  significantly  different  from  the 
performance  of  the  aircraft-trained  group  (if 
anything  it  was  slightly  better).  See  table 
(b)  above. 

c.  Author's  Conclusions:  The  study  showed  that 
training  occurs  in  the  simulator  and  that  it  transfers 
very  effectively  to  the  aircraft.  The  cost  benefits 
are  potentially  very  great — estimated  nearly  1  1/2 
million  dollars/year  with  present  output  of  graduates. 

The  BOPTT  training  transfer  ratio  is  1:1  in  the  early 
phases  of  flight  training  but  declines  in  the  latter 
phases  with  an  average  of  approximately  3:2  for  the 
total  flight  program. 

In  more  advanced  training  for  instructor  operators,  the 
100%  transfer  afforded  by  the  direct  substitution  of 
the  BOPTT  training  for  aircraft  training  is  a  striking 
confirmation  of  device  effectiveness. 

Probably  due  to  the  extremely  high  proficiency  of  the 
personnel  used  as  subjects  in  the  skill  maintenance 
study,  the  BOPTT  had  neither  a  positive  nor  negative 
effect  on  the  maintenance  of  skills  though  refresher 
training,  when  substituted  for  the  aircraft. 


simulator  exclusively;  9  controls  flew  an  equal 
number  of  missions  in  the  aircraft  exclusively. 

All  were  then  evaluated  in  a  single  real-aircraft 
flight  mission. 

In  the  third  study,  experimental  subjects  who  were 
experienced  operators  received  refresher  training 
for  skill  maintenance  in  place  of  aircraft  flight; 
the  controls  continued  their  normal  flight  duties 
without  simulated  training. 

b.  Results: 

(1)  Key  Data: 

(a)  Study  I:  Aircraft  Refueling  Contact 
Attemps  to  Achieve  Proficiency:  All  Groups 


Total 

Total 

Group 

N 

Range 

Mean 

Control 

16 

49-126 

71.06 

El 

13 

41-73 

53.38 

E2 

15 

41-83 

53.60 

E3 

15 

25-88 

50.00 

(b)  Study 

II:  Instructor  Boom  Operator 

Evaluation 

Check 

Ride  Results 

Group 

N 

Range 

Mean 

SD 

Control 

9 

74.4-100 

96 .04 

8.63 

Experimental  12 

88.0-100 

96.37 

3.98 

(2)  Verbal  Description:  No  data  are  given  from 
Study  III  because  it  was  found  that  all  operators 
in  the  study  scored  100%  regardless  of  group,  in 
an  initial  checkride  and  two  in-process  checkrides 
60  and  120  days  later:  no  differential  training 
effects  were  to  be  observed. 

(a)  Study  I:  as  shown  in  table  (a)  above,  at 
the  5%  level  of  significance  the  experimental 
groups  reached  criterion  reliabily  sooner 
than  the  control  group.  The  Boom  Operator 
Progress  Evaluation  scores  found  the 
experimental  groups  significantly  superior  to 
the  control  group  in  the  early  going  (before 
flightline  training) ,  but  by  the  end  of 
flightline  training,  there  was  no  significant 
difference  in  performance  between 
experimental  and  control  groups.  This  rough 
equality  was  achieved  by  the  simulator  groups 
with  40%  fewer  trials  in  the  aircraft  than 
the  controls  had.  Subjective  interview 
responses  from  instructors  agreed  that  the 
device  improved  the  skills  of  students. 

A  cost  analysis  was  also  conducted  which 
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(2)  Study  II:  intermediate 

(3)  Study  III:  expert 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  of  attitude: 

(1)  Instructors:  very  favorable  as  indicated  by 
questionnaire  (Study  I) 

(2)  Students:  not  discussed 

m.  User  of  instructional  features: 

(1)  Intensity:  not  specified,  presumably 
intensive 

(2)  Features  used:  not  specified 
9.  Abstract: 

a.  Study  Synopsis:  The  evaluation  of  the  Boom 
Operator  Part  Task  Trainer  (BOPTT) ,  a  high  fidelity 
simulator  of  an  aircraft  refueling  boom  operation,  was 
divided  into  three  sub-studies  with  the  following 
obj  ectives : 

(1)  Study  I:  to  determine  the  transfer  of 
training  from  the  BOPTT  to  the  KC-135A  aircraft 
for  the  novice  boom  operator  student,  and  to 
evaluate  the  cost-  effectiveness  of  the  BOPTT  in 
the  training  program; 

(2)  Study  II:  to  evaluate  the  BOPTT  effectiveness 
in  training  qualified  operators  to  become 
instructors,  when  the  BOPTT  serves  as  the  sole 
training  device; 

(3)  to  investigate  the  effectiveness  of  the  device 
in  maintaining  skill  for  experienced  personnel, 
when  it  is  substituted  for  aircraft  practice. 

In  the  first  study,  three  experimental  groups  of 
student  operators  unfamiliar  with  the  task  were 
given  progressively  greater  amounts  of  simulator 
training  prior  to  flightline  (actual  aircraft) 
phase  of  training;  one  of  these  groups  was  given 
additional  simulator  training  during  the 
flightline  phase.  The  control  group  received  no 
simulator  training  whatsoever.  Numbers  in  each 
group  were:  experimental,  13,  15,  and  13; 
controls,  15.  Measurements  were  taken  of  number 
of  trials  to  criterion,  and  of  proficiency  in  the 
aircraft,  scored  by  instructors  and  checkride 
observers. 

In  the  second  study,  qualified  boom  operators  were 
being  raised  to  the  instructor  level.  12 
experimental  subjects  were  trained  in  the 
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(2)  Controls: 


9  same 


(1)  Subjects: 

(2)  Subjects: 

(3)  Controls: 


Study  HI 

6  very  experienced  boom  operators 
6  same 
6  same 


c. 


Tests  or  tr 

(1)  Study 
(aircraft 

(2)  Study 

(3)  Study 
checkr ides 


ials/timing 

I:  3  in-process  tr 

checkride) 

II:  1  post-test  (a 

III:  1  in-process 

) 


ials;  1  post-test 

ircraft  checkride) 
tests  (aircraft 


d.  Number  of  different  types  of  measures:  2 
quantitative  for  Study  I;  1  quantitative  Studies  II 
and  III;  interviews  with  instructors,  Study  I 


e.  Description  of  Measurements  and  Ratings 
(quantitative) 

(1)  Number  of  trials  to  proficiency  (Study  I) 

(2)  "Boom  Operator  Progress  Evaluation", 
instructor-  scored,  graded  by  individual  skill 
item  (all  studies) 


f.  Experimental  setting/training  context:  Study  I, 
institutional,  hands-on;  Study  II,  same;  Study  III, 
field,  laboratory,  hands-on 

g.  Statistical  methods:  Chi-square  analysis; 
randomized  group  ANOVA;  Scheffe's  criterion;  two-way 
ANOVA;  t-test;  post  hoc  power  analysis  (Cohen,  1977) 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  above,  section  6 

The  chief  variable  was  the  time  spent  on  device 
during  training.  However,  the  results  show  little 
or  no  differences  between  the  experimental  groups 
in  relation  to  time  on  device. 

(2)  Fidelity  Levels: 

(a)  Physical:  high 

(b)  Functional :  high 

(3)  Type  of  Task/ i... ill  required:  boom  operations; 
perceptual;  psychomotor;  procedural;  part-task 

(4)  Task  difficulty:  high 

i.  Stage  of  Training: 

(1)  Study  I:  introduction,  skill 

(2)  Study  II:  transition,  skill 

(3)  Study  III:  expert,  skill 

j.  Trainee  sophistication 

(1)  Study  I:  novice 


1.  Author:  Gray,  Thomas  H. 


2.  Title:  Boom  Operator  Part-Task  Trainer:  Test  and 
Evaluation  of  the  Transfer  of  Training 

3.  Source:  AFHRL-TR-79-37 ,  October  1979 

4.  Topic  Keywords:  Cost-avoidance  ;  Part-task  trainer  ; 
Transfer  of  training  ;  Skill  maintenance  ;  Simulation  . 

5.  Short  Summary:  This  study  evaluates  a  high-fidelity 
part-task  training  simulator  for  aircraft  refueling  boom 
operation.  The  simulator  was  more  effective  than  the  real 
aircraft  for  training  inexperienced  students,  and  equally 
effective  at  training  more  experienced  operators.  Results 
for  skill  maintenance  refresher  training  of  highly 
experienced  operators  were  inconclusive,  although  the 
simulator  in  this  case  did  not  prove  inferior  to  the  real 
aircraft. 

6.  Devices: 

a.  KG-135A  aircraft  inflight  refueling  boom 

b.  Boom  Operator  Part  Task  Trainer  (BOPTT) , 
high-fidelity  ground-based  simulator  of  flight 
refueling  operation 

7.  Institutions: 


a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 


b.  Performing  Organization:  Flying  Training  Division, 
AFHRL ,  Williams  AFB,  AZ  85224 

8.  Experiment  (Three  Sub-Studies:  I,  II,  III) 

a.  Number  of  Groups:  Study  I,  4;  Study  II,  2;  Study 
III,  3 


b. 


Description  of  Groups: 

iLtudy  I 

(1)  Subjects:  13  student  operators,  matched 
groups  by  academic  grades  and  initial  air 
refueling  block 

(2)  Subjects:  15  same 

(3)  Subjects:  13  same 

(4)  Controls:  15  same 


Study  II 

(1)  Subjects:  12  qualified  boom  operators 
training  to  become  instructors 


among 


Pk 
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Several  limitations  of  this  study  constrain  the 
generality  of  the  results.  First,  subjects  were 
compared  after  only  one  simulated  or  live  tour; 
repeated  exposures  might  produce  larger  performance 
differences.  Second,  both  tour  and  film  groups 
received  passive  rather  than  active  environmental 
experience.  Third,  the  study  examined  only  one  simple 
mode  of  environmental  simulation:  a  continuous  film, 
without  cuts  or  pans,  taken  from  a  single  point  of 
view. 


1.  Author:  Grimsley,  Douglas  L. 


2.  Title:  Acquisition,  Training,  and  Retraining:  Training 
Category  IV  Personnel  With  Low  Fidelity  Devices 

3.  Source:  HumRRO  Technical  Report  69-12,  1969 

4.  Topic  Keywords:  Category  IV  Personnel  , 

Low  Aptitude  K15 ; 47H ; 15rl5 ;H  Personnel  ;  Procedural  Tasks  ; 
Fidelity  ;  Low  Fidelity  ;  Retention  ;  Retraining  ; 
Aptitude  Level  ;  Fixed  Procedure  . 

5.  Short  Summary:  Low  aptitude  personnel  trained  on 
devices  of  differing  fidelity  show  no  significant  difference 
among  themselves  due  to  fidelity  in  acquisition  and 
retention  of  training.  However,  their  time  to  learn 
procedural  tasks  is  significantly  greater  than  for  higher 
aptitude  personnel. 

6.  Devices:  Three  simulated  display  and  control  panels  of 
differing  fidelity  -  High,  Medium,  and  Low 

7.  Institutions: 

a.  Sponsor:  Chief  of  Research  and  Development, 
Department  of  the  Army 

b.  Performing  Organization: 

George  Washington  University 

Human  Resources  Research  Office,  Division  3,  Presidio 
of  Monterey,  CA  93940 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups: 

(1)  Two-Groups  Post-Test  with  follow-up  tests 

(2)  Three  Sub-groups  within  each  group 

b.  Description  of  Groups: 

(1)  Subjects:  72  Army  trainees,  sex  not 
specified,  36  Low  Aptitude  (Category  IV)  and  36 
Middle  and  High  Aptitude 

(2)  Controls:  None 

c.  Tests  or  Trials/Timing:  Post-tests  (over  a  period 
of  6  weeks) 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Time  to  train 

(2)  Proficiency  scores,  scored  by  instructors, 
based  on  number  of  procedural  tests  performed 
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correctly,  regardless  of  time  taken 
(3)  Time  to  retrain 

f.  Experimental  Setting/Training  Context:  School 

g.  Statistical  Methods:  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  "Hot  Panel"  (High  Fidelity)  physical 
duplicate  of  tactical  panel  in  which  all 
lights,  meters,  intercom,  and  other 
indicators  functioned 

(b)  "Cold  Panel",  identical  to  above  except 
no  electric  power 

(c)  "Reproduced  Panel",  full-size  artist's 
rendering  of  the  Hot  Panel. 

(2)  Fidelity  Levels:  High,  Medium,  Low 

(3)  Type  of  Task/Skill  Required:  92-step 
Procedural  Task 

(4)  Task  Difficulty:  Moderate  (simple  but  long) 

i.  Stage  of  Training:  Advanced  Individual  Training 

j.  Trainee  Sophisticatiun:  Low 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  This  study  extends  the  work  of 
HumRRO  Work  Unit  STRANGER  in  examining  acquisition  and 
long-  term  retention  of  motor  skills  in  procedural 
tasks,  using  training  devices  of  varying  degrees  of 
simulation  fidelity.  The  general  conclusions,  that  low 
fidelity  devices  are  quite  suitable  and  therefore 
highly  economical  training  aids  for  procedural  tasks, 
are  substantially  the  same  as  those  arrived  at  in  the 
earlier  reports  by  Grimsley,  HumRRO  Technical  Reports 
69-1  and  69-4.  The  present  study  addresses  the 
question  of  training  low-aptitude  personnel  in  a 
lengthy  procedural  task. 

To  compare  the  performances  of  high  and  low-aptitude 
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trainees  on  simulators  of  varying  fidelity,  72  trainees 
were  divided  equally  into  two  main  groups  (High 
Aptitude  and  Low  Aptitude,  as  measured  by  AFQT  scores 
with  a  dividing  line  at  30)  who  were  to  learn  the 
92-step  procedural  task.  Each  main  group  was 
subdivided  into  three  sub-groups,  trained  respectively 
on  High,  Medium,  and  Low  Fidelity  devices — "Hot," 

"Cold, "  and  "Reproduced"  panels  simulating  a  piece  of 
Nike-Hercules  control  equipment. 

As  with  earlier  studies,  the  fidelity  of  the  training 
panel  had  little  effect  on  training  time.  Fidelity  did 
have  a  slight,  though  for  training  purposes  not 
practical,  effect  on  initial  test  performances. 
(Reviewer's  Note:  The  superior  training  effect  of  the 
High  Fidelity  device  on  the  initial  test  scores  was 
more  pronounced  among  lower-aptitude  subjects.) 

Although  large  and  significant  differences  in  training 
and  retraining  time  was  found  between  High  Aptitude  and 
Low  Aptitude  subjects,  most  of  the  low  aptitude 
subjects  could  be  trained  and  retrained  to  criterion 
within  the  training  time  alloted  under  the  program.  In 
general,  "the  low  aptitude  subjects  were  slower  to 
respond,  required  more  training  time  to  attain  a 
specified  criterion,  needed  more  guidance  and 
repetition  of  instruction,  and  were...  more  variable 
as  a  group  then  the  middle  and  high  aptitude  subjects.' 
The  low  aptitude  person  can  learn  to  perform  a  variety 
of  tasks,  however,  if  the  training  methods  have  been 
carefully  selected  and  organized  to  facilitate  his 
assimilation  of  the  instruction." 

b.  Results: 

Mean  Scores  (Number  Correct)  Transfer 

By  Aptitude  and  Training  Condition 

Immediate  Retention  Four-Week  Retention 


High  AFQT 

Hot  Panel  90.9  75.7 
Cold  Panel  89.2  75.0 
Repro  Panel  88.3  75.0 
Low  AFQT 

Hot  Panel  90.5  72.4 
Cold  Panel  88.0  73.0 
Repro  Panel  86.5  72.9 


On  Immediate  Retention  test,  superiority  of  Hot  Panel- 
trained  groups  is  significant  (p-.001).  Other  effects 
not  significant. 

c.  Authors'  Conclusions:  The  fidelity  of  devices  used 
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to  train  procedural  tasks  can  be  very  low  with  no 
adverse  effect  on  time  to  train,  level  of  proficiency, 
amount  remembered  over  time,  or  time  to  retrain.  This 
applies  to  both  individual  and  group  training 
situations,  and  holds  regardless  of  aptitude  as 
measured  by  AFQT  scores.  Training  device  selection 
should  be  based  on  a  careful  review  of  the  tasks  to  be 
taught  in  order  to  employ  inexpensive  (i.e.,  low 
fidelity)  devices  where  possible. 


1.  Author:  Grunwald,  Walter 


2.  Title:  An  Investigation  of  the  Effectiveness  of 
Training  Devices  with  Varying  Degrees  of  Fidelity 

3.  Source:  Doctoral  dissertation  submitted  to  graduate 
faculty,  Oklahoma  University  at  Norman,  1968;  University 
Microfilms,  Ann  Arbor,  MI,  Order  No.  68-13,  559 

4.  Topic  Keywords:  Homogeneous  Grouping  ;  Fidelity  ; 
Optimal  Fidelity  ;  Low  Fidelity  . 

5.  Short  Summary:  Trainees  who  learned  a  procedural  task 
on  devices  of  five  different  grades  of  fidelity  showed 
differences  in  learned  proficiency  apparently  dependent  on 
the  involvement  and  effort  of  the  learners  rather  than  on 
the  fidelity  of  the  devices.  Results  suggested  that 
training  effectiveness  may  not  increase,  and  may  actually 
decrease,  when  fidelity  is  increased  beyond  an  optimal 
level . 


6.  Devices: 

a.  Actual  aircraft  engineer's  instrument  panel,  fully 
functional 

b.  Simulator  with  functional  controls  and  displays 

c.  Mock-up  with  functional  controls  but  without 
displays 

d.  Full  size  photograph  of  equipment  without  either 
functional  controls  or  displays 

e.  Small  illustration  of  equipment 

7.  Institutions: 


8 . 


a.  Sponsor:  Oklahoma  University,  Norman,  OK  73069 

b.  Performing  Organization:  not  applicable 
Type  of  Article:  Experiment 

a.  Number  of  Groups:  5 


b.  Description  of  Groups: 

(1)  Subjects:  25  randomly  selected  Air  Force 
Flight  Engineer  Technicians  with  high  technical 
aptitude,  and  with  no  prior  experience  on  study 
equipment  or  similar  equipment. 

( 2 )  — ( 5 )  Subjec  s:  25  same 
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c.  Tests  or  Trials/Timing:  3  post-test 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Written  multiple-choice  test  concerning  system 
principles  and  functions 

(2)  Performance  test,  to  complete  procedural  task 
under  routine  conditions 

(3)  Performance  test  requiring  rectifying 
malfunctions 

f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 


g.  Statistical  Methods:  ANOVA 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Actual  aircraft  engineer's  instrument 
panel,  fully  functional 

(b)  Simulator  with  functional  controls  and 
displays 

(c)  Mock-up  with  functional  controls  but 
without  displays 

(d)  Full  size  photograph  of  equipment  without 
either  functional  controls  or  displays 

(e)  Small  illustration  of  equipment 

(2)  Fidelity  Levels:  Physical  Functional 


Device  a 
Device  b 
Device  c 
Device  d 
Device  e 


High 

Medium-high 

Medium 

Medium 

Low 


High 

High 

Medium-high 

Medium 

Medium-low 


Note:  These  assessments  of  fidelity  level  are 

interpretations  by  the  reviewer  and  are  not  fully 
congruent  with  the  author's  interpretation.  A 
portion  of  this  study  is  devoted  in  part  to 
developing  a  means  to  describe  functional 
f idelity . 

(3)  Type  of  Task/Skill  Required:  operations; 
cognitive;  psychomotor;  perceptual;  procedural; 
par t-task 

(4)  Task  Difficulty:  medium 


i.  Stage  of  Training:  procedural,  familiarization, 
skill,  advanced 


j.  Trainee  Sophistication:  intermediate 


k. 


Incorporation  of  Device  into  P.O.I.:  lock-step 


l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  varied;  not 
specified 

Abstract : 


a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
evaluate  the  training  effectiveness  of  five  selected 
training  devices  with  varying  degrees  of  fidelity,  in 
the  learning  of  a  specific  psychomotor,  procedural 
task — a  portion  of  a  flight  engineer's  preflight 
aircraft  check.  Five  groups  of  25  trainees  each — all 
of  whom  already  possessed  basic  ratings  as  Air  Force 
Flight  Engineer  Technicians  and  who  were  the  high 
aptitude — were  trained  on  the  five  different  devices, 
which  ranged  from  actual  aircraft  cockpit  equipment 
(maximum  fidelity) .  Careful  control  was  exercised  to 
keep  all  independent  variables  constant,  aside  from  the 
devices  themselves;  a  preliminary  study  using  32 
students  had  been  performend  to  validate  the  control 
over  the  other  components  of  the  Program  of 
Instruction. 

b.  Results: 


(1)  Key  Data: 


Differences  Between  Means  of 
Performance  Test  Scores 


Group 

Mean 

I 

24.56 

II 

24.20 

III 

25.64 

IV 

23.72 

V 

22.08 

Table  of  Differences 


I 

II 

III 

IV 


II  III  IV  V 

0.36  1.08  0.84  2.48** 

1.44*  0.48  2.12** 

1.92**  3.56** 

1.64** 


**  Significant  on  the  1  per  cent  level,  based  on 
critical  difference  d  =  1.85  *  Significant  on  the 
5  per  cent  level,  based  on  critical  difference  a  = 


(2)  Verbal  Description:  There  was  a  significant 
difference  among  the  five  groups  in  their  ability 
to  perform  the  task  under  routine  conditions.  The 
performance  of  the  group  which  had  received 
training  on  the  mock-up  was  best,  while 
performance  of  the  group  which  had  been  trained 
with  the  aid  of  the  small  illustration  was  worst. 
Performance  of  the  groups  assigned  to  the 
remaining  devices  was  at  approximately  the  same 
level  with  each  other. 

An  increase  in  fuctional  and  appearance  fidelity 
beyond  a  certain  point  was  associated  with  lowered 
effectiveness;  low  effectiveness  also  was 
associated  with  reduction  of  fidelity  to  a  low 
value . 

c.  Authors'  Conclusions:  A  training  device  of 
relatively  moderate  fidelity  was  not  only  more 
effective  than  devices  of  lower  fidelity,  but  was  also 
superior  to  devices  of  very  high  fidelity,  at  least  in 
the  context  of  this  study  with  its  emphasis  on  a 
procedural  task.  The  present  study  suggests  that  a 
plateau  of  learning  as  a  function  of  simulator  fidelity 
is  reached  much  sooner  than  has  been  commonly  assumed, 
and  in  addition,  that  increasing  fidelity  beyond  a 
certain  optimum  value  may  lower  the  effectiveness  of 
the  device. 

A  quantitative  model,  showing  the  relationship  between 
learner  effort  and  device  effectiveness,  helps  to 
demonstrate  the  hypothesis  that  device  effectiveness 
depends  on  the  ability  of  the  device  to  involve  the 
learner  in  the  learning  process,  rather  than  on 
fidelity  alone. 

It  may  be  possible  that  contradictory  research  findings 
in  the  area  of  training  device  fidelity  are  due  not  to 
faulty  experimental  design,  but  to  the  present 
inability  to  state  some  fundamental  laws  governing 
design  of  training  devices  with  respect  to  fidelity. 


1.  Authors:  Hagin,  William  V.  ,  Durall,  Edwin  P.  ,  & 
Prophet,  Wallace  W. 

2.  Title:  Transfer  of  Training  Effectiveness  Evaluation: 
U.S.  Navy  Device  2B35 

3.  Source:  Seville  Technical  Report  TR  79-06,  July  1979 

4.  Topic  Keywords:  Flight  Simulation  ; 

Transfer  of  Training  Weapons  Delivery  Training  ; 

Training  Effectiveness  . 

5.  Short  Summary:  A  transfer  of  training  evaluation  of  a 
high  fidelity  flight  simulator  in  advanced  transitional  jet 
pilot  training  showed  it  effective  in  familiarization  and 
weapons  delivery  training,  but  ineffective  in  night  flying 
training,  and  of  unproved  effectiveness  in  the  highly 
advanced  phase  of  carrier  qualification. 

6.  Devices: 

a.  TA-4J  jet  training  aircraft  b. 

2B35  visual  display  system  integrated  with  the 
2F90  operational  flight  trainer 

7.  Institutions: 

a.  Sponsor:  Chief  of  Naval  Education  &  Training  Code 
N-4,  Pensacola  Naval  Air  Station,  Pensacola,  FL  32508 

b.  Performing  Organization: 

Seville  Research  Corporation,  400  Plaza  Building, 
Pensacola,  FL  32505 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  8  total  combined  in  3  separate 
studies 

(1)  Study  1:  Familiarization,  3  groups 

(2)  Study  2:  Weapons,  2  groups  (6  subgroups) 

(3)  Study  3:  Carrier  Qualification,  3  groups  (6 

subgroups ) 

b.  Description  of  Groups: 

(1)  Subjects:  Study  1 

(a)  20  advanced  jet  carrier  pilot  trainees 
(heavy  simulator  training) 

(b)  20  same  (moderate  simulator  training) 

(c)  20  same  (no  simulator  training) 

(2)  Subjects:  Study  2 

(a)  30  advanced  jet  carrier  pilot  trainees 
(heavy  training) 


(b)  30  same  (not  simulator  training) 

(3)  Subjects:  Study  3  (Same  as  Study  1) 

c.  Tests  or  trials/timing:  Daily  in-process  trials, 
in  simulator  and  in  aircraft 

d.  Number  of  different  types  of  measures:  3 

e.  Description  of  measurements  and  ratings: 

(1)  "Objective  Data"  scores:  instructor-scored 
deviations  from  prescribed  maneuver  parameters 

(2)  Practice  bomb  scores:  circular  error 

(3)  Landing  signal  officer  grades  (data  did  not 
yield  reliable  results) 

f.  Experimental  setting/training  context: 
institutional,  hands-on 

g.  Statistical  methods:  univariate  and  multivariate 
analysis  of  variance,  covariance,  and  regression; 
t-test 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  in  section  6  above 

(2)  Fidelity  levels  (of  simulator): 

(a)  Physical  -  high 

(b)  Functional  -  low 

(3)  Type  of  task/skill  required:  operations; 
cognitive,  psychomotor,  perceptual,  procedural, 
whole- task 

(4)  Task  Difficulty:  high 

i.  Stage  of  training:  familiarization,  skill, 
transition,  advanced 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude: 

(1)  Instructors:  moderately  favorable 

(2)  Students:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  intensive 

(2)  Features  used:  Freeze  capability; 

Restar t/'resequence  capability;  Number/quality  of 
responses;  others  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  This  report  presents  results  of  a 
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transfer  of  training  effectiveness  evaluation  of  a 
visual  flight  simulator,  Navy  Device  2B35,  a 
computer-generated  imaging,  wide  angle  visual 
attachment  to  an  operational  2F90  flight  trainer  being 
used  in  advanced  jet  training.  The  experiment  was 
divided  into  three  main  parts,  investigating  the  use  of 
the  device  in  (1)  Familiarization,  and  Night 
Familiarization;  (2)  Weapons;  (3)  Carrier 
Qualification.  A  total  of  60  Advanced  Jet  student 
pilots  were  subjects  of  the  study.  In  part  1  of  the 
experiment,  they  were  divided  into  three  groups  of  20, 
in  which  one  group  received  aircraft  training  and  no 
device  training  and  the  other  two  received  no  aircraft 
training  and  different  amounts  of  device  training  prior 
to  their  first  aircraft  flight;  in  part  2,  Weapons, 
the  trainees  were  divided  into  2  groups  of  30  each,  in 
which  one  group  received  no  device  training  while  the 
other  did;  part  3  was  structured  the  same  as  part  1. 

Objective  measurements  taken  during  trials  in  the 
simulator,  and  later,  trials  in  the  aircraft,  were  (1) 
deviations  from  prescribed  maneuver  parameters,  and  (2) 
circular  error  in  practice  bombing  (weapons  phase 
ff)  •  only).  Subjective  instructor  grades  were  also  given 

for  aircraft  flights. 

b.  Results: 

(1)  Key  Data:  Mean  Percent  Error  By  Group  And 


Flight:  WEP  Delivery 

Patte 

rn 

Group 

5 

Flight 

6 

7 

Simulator- trained 

39% 

30% 

39% 

Non-Simulator- trained 

45% 

38% 

38% 

Reviewer's  Note:  These  figures  are  approximations 
from  a  graphic  representation  in  the  report. 

(2)  Verbal  Description:  No  significant  difference 
in  transfer  performance  resulted  from  device 
training  in  the  carrier  qualification  phase, 
although  missing  data  make  this  report 
inconclusive.  For  the  Night  Flying 
Familiarization  transfer  task,  the  only 
significant  differences  which  appeared  indicated 
superiority  for  the  group  which  had  no  device 
training . 

Results  in  the  weapons  delivery  transfer 
performances,  and  in  day  familiarization,  however, 
did  show  simulator  training  equal  to  or  even 
superior  to,  training  in  the  aircraft.  Mean 
percent  error  in  flight  patterns  for  the  weapons 
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aircraft  trials  are  shown  in  the  table  above.  The 
MVF  for  the  weapons  main  treatment  effect  was  2.63 
with  10  and  47  df;  p<.02. 

In  the  tests  of  transfer  performance  in  the 
familiarization  phase#  few  significant  differences 
appeared  among  many  performance  measures  between 
treatment  groups,  indicating  that  device  training 
was  as  effective  as  aircraft  training. 

c.  Authors'  Conclusions:  The  transfer  results 
presented  do  provide  support  for  the  continued 
utilization  of  the  2B35  in  Navy  Advanced  Jet  training. 
However,  such  support  requires  qualification  with 
reference  to  the  skills  and  tasks  to  be  taught.  Those 
results  support  the  use  of  the  device  for 
f amilizarization  maneuvers,  but  not  without  some 
qualifications.  On  the  other  hand,  the  results  offer 
no  support  to  the  use  of  the  device  for  night 
familiarization  instruction.  The  clearest  support  for 
2B35  utility  is  in  the  weapons  training  area,  but  no 
conclusion  can  reasonably  be  drawn  in  favor  of  use  of 
the  device  in  the  carrier  qualification  phase  of 
training. 
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1. 


Author:  Hagman,  Joseph  D. 


2.  Title:  Effects  of  Training  Task  Repetition  on  Retention 
and  Transfer  of  Maintenance  Skill. 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Research  Report  1271,  May 
1980  . 

4.  Topic  Keywords:  Task  Repetition  ;  Retention  ; 

Training  Transfer  ;  Maintenance  Skill  . 

5.  Short  Summary:  This  eperiment  investigated  the  effects 
of  task  repetitions  on  the  performance  of  a  simple  task. 

The  task  repetition  levels  varied  from  no  (0)  repetitions  to 
four  (4)  repetitions  of  the  task.  An  apparent  ceiling  of 
three  repetitions  of  the  tasks  during  training  was  found  to 
enhance  both  acquisition  of  skill  and  2-weeks  retention  of 
skill  in  task  performance.  However,  the  task  repetitions 
did  not  enhance  transfer  of  skill  to  other,  similar 
equipment . 

6.  Device:  500A  Sun  Test  Stand  (Actual  Test  Equipment) 

7.  Institutions: 

a.  Sponsor:  Deputy  Chief  of  Staff  for  Training,  Army 
Training  and  Doctrine  Command,  Fort  Eustis,  VA  23604. 

b.  Performing  Organization:  Army  Research  Institute 
for  the  Behavioral  and  Social  Sciences,  Army  Training 
and  Doctrine  Command,  Fort  Eustis,  VA  23604. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  5 

b.  Description  of  Groups:  Subjects  were  military  fuel 
and  electrical  repairmen 

(1)  15,  trained  by  4  task  repetitions 

(2)  15,  3  task  reps 

(3)  15,  2  task  reps 

(4)  15,  1  task  rep 

(5)  Controls:  15  same,  0  task  reps 

c.  Tests  or  Trials/Timing:  3  post  tests:  1 
immediately  for  subjects  only;  1  delayed  for  subjects 
only;  1  delayed  transfer- to-other-equipment  for 
subjects  and  controls 

d.  Number  of  different  types  of  measures:  2 
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e.  Description  of  Measurements  and  Ratings: 

(1)  Time  to  complete  tasks 

(2)  Number  of  errors  in  performing  tasks 

f.  Experimental/Training  Context:  unspecified; 
apparently  laboratory/  hands-on 

g.  Statistical  Methods:  One  way  ANOVA;  mixed 
factorial  ANOVA 

h.  Variables  being  manipulated: 

(1)  Training  Device:  500A  Sun  Test  Stand; 
however,  number  of  task  repetitions,  not  device, 
was  the  variable  of  interest 

(2)  Fidelity  Levels: 

(a)  Physical;  exact — standard  equipment 

(b)  Functional:  same 

(3)  Type  of  Task/Skill  required:  maintenance 
checking  psychomotor;  cognitive;  perceptual, 
procedural,  part-task 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  procedural,  familiarization 

j.  Trainee  Sophistication:  novice  and  intermediate 
(students,  but  amount  of  prior  training  unspecified) 

.  Incorporation  of  device  into  P.O.I.:  unspecified 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  specified 
9.  Abstract: 

a.  Study  Synopsis:  The  general  purpose  of  this  study 
was  to  provide  information  concerning  the  effects  of 
task  repetition  on  the  retention  and  transfer  of 
AIT-acquired  maintenance  skill. 

75  student  fuel  and  electrical  repairmen  were  randomly 
selected  and  divided  into  five  groups  of  15  each.  One 
group  served  as  a  control  for  only  the  post-experiment 
delayed  transfer  test,  and  received  only 
familiarization  but  no  training  on  the  device  (a  500A 
Sun  Test  Stand,  actual  maintenance  testing  equipment). 
The  other  four  groups  received,  respectively,  1,  2,  3, 
and  4  training  trials  for  5  maintenance  tasks,  testing 
a  100-amp  alternator  with  the  device.  The  4 
experimental  groups  were  subsequently  tested  for  speed 
and  accuracy  of  performance  of  the  5 

maintenance-checking  tasks,  immediately  after  training, 
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f.  Experimental  setting/training  context:  Laboratory, 
hands-on 

g.  Statistical  methods:  Multivariate  analysis  of 
variance 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  above.  Four  simulated 
variables  were  manipulated: 

(a)  Motion:  none,  3  DOF,  6  DOF 

(b)  G-seat:  nonoper ational ;  intermediate; 
fully  operational 

(c)  Field  of  view:  Full  capability, 
intermediate  capability,  narrow  view 

(d)  Ceiling/visibility:  clear  and  unlimited; 
minimum  and  limited  ceiling 

(2)  Fidelity  levels: 

(a)  Physical:  High,  with  variability,  in 
that  f ield-of-view  and  ceiling/visibility 
were  being  manipulated 

(b)  Functional:  High,  with  var.ability,  in 
that  G-  seat  and  platform  motion  were  being 
manipulated 

(3)  Type  of  task/skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  procedural, 
part-task 

(4)  Task  difficulty:  medium-high  to  high 

l.  Stage  of  training:  skill  (not  actually  a  training 
exercise) 

].  Trainee  sophistication:  expert 

k.  Incorporation  of  device  into  P.O.I.:  not 
applicable 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  intensive 

(2)  Features  used:  Cue  enhancement  features; 
Number/quality  of  responses;  plus  others  not 
specif  led 

9.  Abstract: 

a.  Study  Synopsis:  This  study  constituted  a  follow-up 
investigation  of  the  effect  on  experienced  pilot 
performance  of  manipulating  system  variables  within  the 
Advanced  Simulator  for  Pilot  Training.  The  larger 
purpose  of  this  pair  of  studies  was  to  determine  the 
relative  training  benefits  of  different  variables.  It 
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1.  Authors:  Irish,  Philip  A.,  &  Buckland,  George  H. 

2.  Title:  Effects  of  Platform  Motion,  Visual  and  G-seat 
Factors  upon  Experienced  Pilot  Performance  in  the  Flight 
S imulator 

3.  Source:  AFHRL-TR-7 8-9 ,  June  1978 

4.  Topic  Keywords:  Flight  Simulation  ;  Platform  Motion  ; 
Field-of-View  ;  G-seat  factors  ;  Pilot  Performance  . 

5.  Short  Summary:  An  investigation  of  the  effects  of  four 
simulated  variables  on  experienced  pilot  performance  in  the 
Advanced  Simulator  for  Pilot  Training  finds  varying  degrees 
of  influence  on  performance,  but  all  system  effects  were 
overshadowed  by  the  differences  between  subjects. 

6.  Devices:  Advanced  Simulator  for  Pilot  Training 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division 
AFHRL ,  Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  groups:  1 

b.  Description  of  groups:  Subjects  were  5  T-37 
Instructor  Pilots  with  300-2,000  total  flying  hours 
(minimum  of  160  in  simulated  aircraft) 

c.  Tests  or  trials/ timing:  in-process  trials 

d.  Number  of  different  types  of  measures:  3 

e.  Description  of  measurements  and  ratings: 

(1)  System  output  measures:  deviations  using  the 
root  mean  square  technique  from  specified  criteri 
within  segments  of  a  maneuver 

(2)  Pilot  input  measures  -  in  the  form  of 
elevator,  aileron  and  rudder  power  for  maneuver 
segments,  an  attempt  to  measure  analogs  of  pilot 
workload 

(3)  Derived  scores:  composite  scores  for  a 
segment  or  combination  of  segments  within  a 
maneuver,  indicating  how  well  pilot  remained 
within  tolerance  limits  of  several  criteria 
simultaneously 


familiarity  was  also  significant,  with  the  most 
improvement  due  to  aided  training  occurring  for 
the  unfamiliar  powerplants. 

c.  Authors'  Conclusions:  There  are  three  particularly 
important  results  to  be  noticed.  (1)  All  groups  of 
trainees  were  suboptimal  in  that  they  used  operations 
that  provided  low  information  gain  at  high  cost.  (2) 
Fourth-semester  trainees  had  a  significant  positive 
transfer  of  aided  context-free  training  to  context- 
specific  tasks.  (3)  Aided  training  is  of  most  use  for 
transferring  to  unfamiliar  problems. 

A  consistent  interpretation  of  these  and  previous 
studies  leads  one  to  conclude  that  context-free 
training  may  be  more  consistently  valuable  for  trainees 
who  are  farther  along  in  their  training  program. 

The  use  of  context-free  simulations  appears  quite 
promising. 
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a.  Study  Synopsis:  Although  logical  fidelity  is 
presumed  to  be  the  most  important  factor  in  simulator 
design —  particularly  in  the  area  of  troubleshooting 
maintenance —  simulators  are  often  designed  with  more 
emphasis  on  imitating  specific  real-world  appearances 
than  in  developing  general  problem-solving  skills. 

This  study  attempted  to  evaluate  the  training 
effectiveness  of  FAULT,  an  interactive  computerized 
maintenance  troubleshooting  simulator,  in  teaching,  via 
context-free  tasks,  general  problem-solving  skills 
which  could  then  be  applied  to  real-world  context 
specific  troubleshooting  tasks. 

Two  experiments  were  conducted  with  trainees  in  a 
2-year  FAA  certificate  program  in  aircraft  powerplant 
maintenance.  Procedures  and  testing  were  the  same  in 
both  experiments;  however,  the  subjects  and  controls 
in  one  were  26  trainees  in  the  4th  and  final  semester, 
in  the  other  were  60  trainees  in  their  1st  semester. 

All  subjects  were  volunteers.  Their  simulator  training 
consisted  of  three  sessions  of  context-free  problem¬ 
solving,  wherein  half  of  each  group  received 
interactive  aid  from  the  computer  and  the  other  half 
was  unaided.  Testing  consisted  of  solving  context 
specific  problems  in-process  (after  the  2nd  and  3rd 
training  sessions)  and  in  a  final  post-test.  The 
criterion  in  all  tasks  was  to  minimize  the  overall 
costs  of  solutions.  Measures  extracted  from  the  data 
included  not  only  overall  costs  but  also  "fine-grained" 
criteria  quantifying  the  quality  of  the  diagnostic 
strategy. 

b.  Results: 

(1)  Key  Data:  There  was  a  significant  difference 
between  the  average  cost  for  familiar  CAR  problems 
($49)  and  unfamiliar  ($2271)  TPE  and  JT12 
problems —  F ( 1 , 164 ) =41  .  2 8  ,  p-0.01. 

(2)  Verbal  Description:  There  appears  to  be  a 
negative  transfer  of  aided  training  for 
first-semester  trainees  and  a  positive  transfer  of 
aided  training  for  f our th- semester  trainees. 

For  suboptimality  costs  it  was  found  that  the 
group  trained  with  aid  significantly  outperformed 
the  unaided  group.  The  improvement  was  greater 
for  first-semester  trainees  than  for 
fourth-semester  trainees  but  both  were 
significant.  The  interaction  of  aiding  and 


post-test 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Overall  "product"  measures,  e.g.  cost  and 
time-to-  solution 

(2)  Fine-grained  "process"  measures,  e.g., 
"expected  information  gain  per  action, 
suboptimality  (with  respect  to  a  minimal  cost  per 
bit  solution)  due  to  errors  and  inefficiency,  and 
the  allocation  of  expenditures  among  observations, 
bench  tests,  and  unnecessary  replacements." 

f.  Experimental  Setting/Training  Context: 

Institutional  laboratory,  hands-on 

g.  Statistical  Methods:  Three-factor  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  FAULT  interactive 
computerized  troubleshooting  simulator. 

(a)  Aiding  Mode 

(b)  Unaiding  Mode 

(2)  Fidelity  Levels: 

(a)  Physical:  low  (abstract  context-free) 

(b)  Functional:  unknown;  hypothesized 
psychological  fidelity  very  high 
(experimental) 

(3)  Type  of  Task/Skill  Required:  Maintenance 
troubleshooting;  cognitive 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  skill 

j.  Trainee  Sophistication:  intermediate  for  First 
Experiment;  novice  for  Second  Experiment 

k.  Incorporation  of  Device  into  P.O.I.:  Experimental 
self-  paced 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  Resta r t/Resequence  Capability; 
Sign-  in  Capability;  Number/Quality  of  Responses; 
Record/Playback  (a  summary  of  events  rather  than  a 
playback);  perhaps  Internal  Monitoring  of 
Instructional  Features;  perhaps  Next  Activity 
Features 
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1.  Authors:  Hunt,  Ruston  M.  , 


&  Rouse,  William  B. 


«)• 


2.  Title:  Problem-Solving  Skills  of  Maintenance  Trainees 
in  Diagnosing  Faults  in  Simulated  Powerplants 

3.  Source:  Human  Factors  ,  1981,  23(3),  317-328 

4.  Topic  Keywords:  Maintenance  Training  ;  Context-free  ; 
Context-specific  ;  Fault  Diagnosis  Tasks  ; 

Interactive  Computer  Aiding  . 

5.  Short  Summary:  The  real-world  value  of  context-  free 
training  was  evaluated  by  the  testing  of  context-  specific 
troubleshooting  ability  in  subjects  previously  trained  by 
context-free  methods,  with  mixed  practical,  but  promising 
theoretical  results.  These  results  suggest  that 
context-free  training  would  be  more  valuable  to  students 
further  along  in  their  training  programs. 

6.  Devices:  FAULT  interactive  computerized  maintenance 
troubleshooting  simulator. 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
Alexandria,  VA  22333 

b.  Performing  Organization: 

University  of  Illinois,  Urbana,  Illinois  61801 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  4 

b.  Description  of  Groups:  FIRST  EXPERIMENT 

(1)  Subjects:  13  volunteer  powerplant  maintenance 
trainees  in  4th  and  final  semester  of  2-year  FAA 
certificate  program  in  aircraft  powerplant 
maintenance 

(2)  Controls:  13  same 

Note:  numbers  13  are  assumed  by  equal  division  of 

pool  of  26.  Authors  did  not  state  how  the  26  were 
divided. 

Description  of  Groups:  SECOND  EXPERIMENT 

(1)  Subjects:  30  same  as  above  but  in  1st 
semester  of  course 

(2)  30  same  as  subjects,  2nd  experiment 

Note:  same  as  for  1st  experiment 

c.  Tests  or  Trials/Timing:  2  in-process  trials;  1 
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simulation  (not  instructional)  features  of  the  training 
device . ) 


to  overtraining.  This  comparison  points  out 
the  desirability  of  training  to  a  specific 
performance  criterion  rather  than  to  a  time 
schedule . 

(In  addition,  TERs  and  learning  curves  for 
both  groups  were  broken  out  for  each  of  the 
24  maneuvers  under  study.  The  differences  in 
effectiveness  among  various  maneuvers  were 
attributed  to  specific  shortcomings  of  the 
device,  particularly  in  visual  ground 
reference  at  very  low  altitudes  (hovering 
flight) . 

(b)  Experiment  2:  As  shown  in  the  above 
table,  the  simulator  trained  group  improved 
its  performance  significantly  during  the 
training  period  while  the  untrained  group  did 
not.  Furthermore,  the  insignificant 
correlation  between  CH-47  flight  time  during 
the  evaluation  period  and  the  posttest 
checkrides  scores  means  that  the  improvement 
in  performance  of  the  simulator-trained 
aviators  was  due  to  their  training  and  not  to 
experience  in  the  aircraft. 

c.  Author's  Conclusions:  Part  I  on  transition 
training  and  Part  II  on  combat  readiness  flying  both 
show  that  the  CH47FS  is  an  effective  training 
device....  A  small  group  of  maneuvers  had  CTERs  below 
0.7.  However,  CTERs  in  the  range  of  0.5  to  0.7  do  not 
indicate  that  this  training  device  is  ineffective. 
Rather,  they  indicate  that  the  training  device  is  not 
as  efficient  in  terms  of  the  number  of  trials  required 
to  learn  a  particular  maneuver  as  is  the  actual 
aircraft. 

The  maneuvers  that  produced  the  lower  CTERs  were  all 
maneuvers  in  which  a  substantial  part  of  the  maneuver 
was  spent  close  to  the  ground.  It  is  believed  that 
these  difficulties...  are  due  primarily  to  limitations 
in  the  visual  system.  Another  source  of  difficulty  in 
performing  and  training  hovering  maneuvers  could  be  the 
aerodynamic  simulation  of  hovering. 

To  obtain  the  maximum  benefit  from  the  simulator,  it  is 
recommended  that  a  minimum  of  time  be  spent  training 
the ...  maneuver s  with  the  low  CTERs.  The  limited 
simulator  time  should  be  spent  training  those  maneuvers 
with  the  highest  transfer  to  the  aircraft. 

(Further  recommendations  were  made  to  improve  certain 
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b.  Results: 


(1)  Key  Data: 

Experiment  1: 

Cumulative  Transfer  Effectiveness 
Ratio  (CTER)  from  Simulator  to  Aircraft 

CTER  Trials 

to  Criterion  CTER  Time 

Crit.6  Crit.8  Crit.8  Total 
Overall  .69  .82  .95  .70 

(24  maneuvers) 

Experiment  2: 

Keans  and  t-Tests  of  Pretest  and  Posttest 


Checkr ides 
Group 

Scores 

for  Experiment 

and 

Control 

Group 

Pretest 

Posttest  t 

df 

P 

Experimental 

47.5 

56.7  6.8 

14 

<  .002 

Control 

52.5 

53.7  .98 

12 

t 

2.3 

1.0 

df 

26.0 

26.0 

P 

<  .05 

<  .  4 

(2)  Verbal  Description: 


(a)  Experiment  1:  In  reference  to  the 
Transfer  of  Effectiveness  Ratios  shown  above, 
the  formula  was: 

CTER  =  (control  group  aircraft  trials  or  time  -  experimental 
group  aircraft  trials  or  time) /experimental  group  simulator 
trials  or  time 


These  CTERs  express  the  training  effectivenss 
of  the  CF47FS  if  used  in  a  training  program, 
including  all  of  the  24  tested  maneuvers. 

All  of  the  overall  CTERs  are  conservative 
because  of  overtraining.  The 
trials-to-criter ion  8  CTER  is  higher  than  the 
tr ials-to-criter ion  6  CTER  due  to  the  effect 
of  overtraining  on  the  trials-  to-criterion  6 
CTER.  The  time- to-cr iter ion  8  CTER  is  higher 
than  the  trials-to-criter ion  8  CTER  because 
of  the  general  time  advantage  of  the 
simulator  over  the  aircraft.  The  total  time 
CTER  is  lower  than  the  time-to-cr iter  ion  due 
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m.  User  of  instructional  features: 

(1)  Intensity:  presumably  intensive 

(2)  Features  used:  instructional  features  not 

specif ied 

Abstract : 

a.  Study  Synopsis:  This  paper  reports  on  two 
experiments  undertaken  to  assess  the  training 
effectiveness  of  the  CH47FS,  a  prototype  high-fidelity 
visual  helicopter  flight  simulator  designed  as  a 
training  device  for  the  Chinook  CH-47  helicopter. 

The  first  experiment  studied  the  transf er-of- training 
for  16  student  pilots  who  received  most  of  their 
transition  training  to  the  Chinook  CH-47  on  the 
simulator,  and  some  in  the  aircraft  (the  latter  chiefly 
in  the  form  of  checkf lights) .  The  controls  were  35 
students,  matched  for  aptitude  with  the  subjects,  who 
received  all  of  their  training  in  the  aircraft,  in 
accordance  with  the  standard  course  of  instruction. 

All  students  were  subsequently  given  checkrides  and 
trained  to  qualification  criterion — if  further  training 
was  needed — in  the  aircraft.  Dependent  variables  were: 
time  to  criterion;  trials  to  criterion;  and  checkride 
grades.  Results  were  evaluated  by  means  of  the 
Transfer  Effectiveness  Ratio  developed  by  Roscoe  (1971, 
1972)  . 

The  second  experiment  investigated  the  capability  of 
training  in  the  simulator  to  keep  experienced,  working 
pilots  prepared  for  combat.  16  subjects  and  16 
controls,  FORSCOM  aviators  currently  flying  the  CH-47, 
were  selected  for  the  study  by  company  commanders 
according  to  criteria  unknown  to  the  experimenters. 
Subjects  and  controls  were  given  pretraining  and 
post- training  checkrides  and  rated  per  maneuver. 

During  the  6-month  training  period,  all  subjects  and 
controls  flew  only  mission-essential  flying  in  the 
aircraft  (differences  between  the  mission  flying  times 
for  the  two  groups  during  the  period  were  found 
nonsignificant  by  t-test;  outside  the  aircraft,  the 
subjects  received  a  mean  29.7  hours  training  on  the 
simulator,  while  the  controls  received  no  training. 


c.  Tests  or  Trials/Timing: 

Experiment  1:  In-process  trials;  1  post-test 
(aircraft  checkride) 

Experiment  2:  1  pre-test;  1  post-test  (aircraft 

checkr ides) 

d.  Number  of  different  types  of  measures:  4 

e.  Description  of  measurements  and  ratings: 

(1)  Experiments  1  and  2:  12-point  scale 
performance  rating  by  maneuver  (Instructor  Pilot 
rated) 

(2)  Experiment  1  only:  3-point  ratings  of 
subtasks  (by  Instructor  Pilot) 

(3)  Experiment  1  only:  ‘Number  of  trials  to 
criterion 

(4)  Experiment  1  only:  ‘Time  to  criterion  (flight 
time  in  simulator  and  in  aircraft) 

‘(3)  and  (4)  were  the  dependent  variables  analyzed 
in  Experiment  1 

f.  Experimental  Setting/Training  Context: 

Experiment  1:  Institutional,  hands-on 
Experiment  2:  Laboratory,  hands-on 

g.  Statistical  Methods:  t-test;  others  (if  any)  not 
specified 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  above,  section  6  Note: 
both  simulator  and  aircraft  were  Training  Devices 
for  experimental  subjects 

(2)  Fidelity  Levels:  (Simulator) 

(a)  Physical:  Medium-high  to  high 

(b)  Functional:  High 

(3)  Type  of  Task/Skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  whole-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  training: 

Experiment  1:  introduction,  familiarization, 
skill,  transition 

Experiment  2:  refresher,  skill,  advanced 

j.  Trainee  sophistication: 

Experiment  1:  not  specified 
Experiment  2:  expert 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 
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1.  Author:  Holman,  Garvin  L. 

2.  Title:  Training  Effectiveness  of  the  CH-47  Flight 
Simulator 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Research  Report  1209,  May 
1979 

4.  Topic  Keywords:  Simulation  Fidelity  ; 

Visual  Flight  Simulation  ;  Transfer  of  Training  . 

5.  Short  Summary:  A  transfer  of  training  experiment  using 
inexperienced  student  pilots,  and  a  combat  readiness 
training  experiment  using  experienced  pilots,  showed  a 
helicopter  flight  simulator  effective  in  training  for  all 
flight  maneuvers  except  those  requiring  a  high  degree  of 
visual  ground  referencing. 

6.  Devices: 

a.  Chinook  CH-47  helicopter 

b.  CH47FS  High  Fidelity  Visual  Flight  Simulator 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Aviation  Center,  Directorate 
of  Training  Developments,  Fort  Rucker,  AL  36362 

b.  Performing  Organization:  U.S. 

Army  Research  Institute  for  the  Behavioral  and  Social 
Sciences,  5001  Eisenhower  Avenue,  Alexandria,  VA  22333 

8.  Experiment: 

a.  Number  of  Groups:  Experiment  1:  2 

Experiment  2 :  2 

b.  Description  of  Groups: 

Experiment  1 : 

(1)  Subjects:  24  student  aviators,  matched  with 
controls  by  Flight  Aptitude  Selection  Test  and 
Bennet  Mechanical  Comprehension  Test 

(2)  Controls:  35  same 

Experiment  2: 

(1)  Subjects:  16  experienced  FORSCOM  aviators; 
unmatched — selection  criteria  unknown  to 
experimenters 

(2)  Controls:  16  same 


increased  transfer  with  increased  training  repetition 
may  have  been  due  to  the  lack  of  task  variety 
experienced  during  training.  It  is  possible  that  in 
the  present  experiment,  added  training  task  variety  was 
necessary  for  the  benefits  of  task  repetition  to  be 
observed. 

In  contrast  to  retention,  transfer  performance  was  not 
a  function  of  prior  training  task  repetitions.  The 
lack  of  a  repetition  effect  was  suggested  to  be  a 
function  of  "floor  effects"  adversely  operating  on  the 
transfer  data  or  to  the  need  for  introduction  of  added 
task  variety  during  training  repetitions. 
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and  again  14  days  later  without  training  on  the  device 
in  the  interval.  In  addition,  immediately  after  the 
14-day  retention  test,  they  were  tested  for  performance 
of  the  5  tasks  using  the  same  test  device  but  checking 
a  60-amp  alternator  instead  of  the  100-amp  alternator 
on  which  they  had  been  trained.  This  last  test  was 
also  given  to  the  fifth,  previously-  mentioned  control 
group  who  had  familiarization  with  the  testing  device 
but  no  trials. 


Results 


(1)  Key  Data: 

Time 

to  Complete 

Retentions  Tests 

(in  appx. 

minutes) 

Immediate 

Delayed  (Retention) 

1-rep  group 

19 

27 

2-rep  group 

17 

25 

3-rep  group 

15 

21 

4-rep  group 

16 

16.5 

(The  apparent 

difference 

in  retention  loss  rate 

over  time  for  the  4-rep  group  was  not  reliable  as 
indicated  by  the  nonsignificant  interaction.  It 
could  have  occurred  by  chance.) 


Time  To  Complete  Transfer  Test 
(in  approximate  minutes) 


0-rep  Controls  43.5 

1- rep  group  11 

2- rep  group  15.5 

3- rep  group  14.5 

4- rep  group  14.25 


(2)  Verbal  Description:  Very  similar  results  to 
the  time  measures  shown  above  were  obtained  among 
the  accuracy  measures. 

Only  minimal  hands-on  training  experience  plus 
familiarization  information  were  necessary  to 
produce  effective  transfer  performance. 

In  both  retention  and  transfer  experiments,  the 
most  errors  were  found  in  the  task  segment  which 
placed  the  greatest  emphasis  on  rote  memorization. 

c.  Author's  Conclusions:  Maintenance  task  retention 
improved  in  terms  of  both  speed  and  accuracy  as  the 
number  of  task  repetitions  performed  during  training 
increased.  Delayed  test  performance  was  a  direct 
function  of  the  level  of  original  learning  attained 
during  training.  Reliable  retention  improvements 
occurred  at  the  third  trai.  mg  repetition  with  no  added 
benefit  resulting  from  a  fourth  repetition.  Besides 
possible  "floor  effects"  in  the  data,  the  lack  of 
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is  assumed  that  the  responses  of  expert  pilots  to 
system  variables  can  guide  researchers  in  designing 
experiments  with  less  experienced  pilots  in  training. 

The  immediate  objective  of  this  particular  experiment 
was  to  assess  empirically  the  performance  of 
experienced  T-37  pilots  in  the  simulator  under  varying 
platform  motion,  G-seat,  f ield-of-view,  and 
ceiling/ visibility  configurations. 

Five  experienced  pilots,  fully  qualified  and  current  in 
the  aircraft  being  simulated,  performed  contact  and 
instrument  maneuvers  (aileron  roll,  barrel  roll,  loop, 
360  degree  overhead  pattern,  GCA  maneuver)  in  random 
order  in  the  simulator  under  a  broad  range  of  randomly 
selected  variable  conditions,  each  pilot  performing  a 
total  of  between  14  to  21  maneuvers  in  the  session. 
There  were  a  total  of  54  treatment  combinations,  a 
product  of  (a)  three  levels  of  Platform  Motion  (b) 
three  levels  of  G-seat  oper ationabil ity ,  (c)  three 

levels  of  Field  of  View,  and  (d)  two  levels  of 
Ceiling/Visibility. 

A  large  number  of  measurements  were  taken  by  way  of  the 
device's  Automated  Performance  Measure  System,  which 
fell  into  three  basic  categories:  (a)  system  output 
measures;  (b)  pilot  input  measures;  (c)  derived 
scores,  the  latter  being  composite  scores  which 
indicated  how  well  the  pilot  remained  within  tolerance 
limits  of  several  criteria  simultaneously. 

Data  were  subjected  to  multiple  analysis  of  variance  to 
account  for  all  measured  dependent  variables,  and  to 
detect  interactions  between  the  various  treatment 
conditions. 

b.  Results: 

(1)  Key  Data:  no  tabular  data  were  given  for  the 
subject  effect,  which  was  the  most  important 
effect  found  in  the  study,  according  to  the 
authors. 

(2)  Verbal  description;  per  device  treatment 

(a)  Field-of-View :  produced  significant 
multivariate  effects  in  four  of  the  five 
maneuvers. 

(b)  Motion:  significant  multivariate  motion 
effects  were  demonstrated  in  three  of  the 
five  maneuvers:  loop,  overhead  pattern,  and 
GCA. 

(c)  G-Seat:  was  conspicuous  by  the  complete 
absence  of  significant  multivariate  effects 
in  any  of  the  maneuvers. 
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(d)  Ceiling/visibility  produced 
multivariate  and  univariate  significance  in 
the  two  maneuvers  where  it  was  manipulated, 
with  best  performance  demonstrated  under  the 
clear  condition. 

(e)  The  only  interaction  achieving 
multivariate  significance  was  the  first  order 
platform  motion  by  ceiling  interaction  in  the 
GCA  maneuver;  analysis  of  this  interaction 
shows  all  of  the  five  measures  generally 
associating  superior  performance  with  the 
clear  ceiling/visibility  condition. 

(f)  Subject  effects:  Consistently,  subject 
differences  accounted  for  the  largest  portion 
of  the  performance  variability  on  each 
measure.  This  percentage  ranged  from  15  to 
89  percent  of  the  total  variability  in  the 
performances  of  these  maneuvers. 

(g)  Comparative  strengths  of  effects  other 
than  subject:  Fiela-of-View  and  Motion 
factors  were  highly  variable  from  measure  to 
measure.  The  G-  seat  was  considerably  more 
consistent  across  dependent  measures  and 
across  maneuvers. 

c.  Authors'  Conclusions:  The  data  indicate  that  in 
comparison  to  subject  differences  and  environmental 
factors  (including  ceiling/visibility),  the  design 
variables  are  of  lesser  importance. 

The  fact  that  people  are  different  is  nothing  new  or 
surprising;  however,  it  strongly  suggests  that 
individual  pilots  respond  to  simulator  configurations 
differently  and  disconfirms  the  hypothesis  that  any 
simple  linear  model  of  piloting  behaviors  is  sufficient 
to  describe  the  processes  involved  in  controlling  an 
aircraft. 

The  differences  caused  by  the  motion  variable  were 
often  manifested  in  measurements  particularly  sensitive 
to  pitch  control.  Furthermore,  in  some  cases,  the 
changes  were  more  frequently  recorded  within  the  pilot 
input  category  of  the  dependent  measures,  suggesting 
that  this  variable,  while  often  not  strong  enough  to 
alter  the  overall  performance  of  the  vehicle,  does 
cause  changes  in  the  pilots'  controlling  strategy. 

Both  :he  motion  variable  and  the  f ield-of-view  variable 
together  accounted  for  approximately  1  to  10  percent  of 
the  performance  variability.  The  effects  of  field  of 
view  were  generally  at  least  equal  to  the  effects  of 
motion . 
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The  existence  and  the  nature  of  the  subject  effects  and 
the  system  configuration  variable  effects  have 
demonstrated  the  utility  and  efficacy  of  a 
comprehensive  performance  measurement  strategy  which 
addresses  not  only  the  traditional  system  output 
measures  but  also  includes  control  strategy  measures  in 
the  form  of  pilot  workloads  and  input  smoothness.  The 
results  suggest  that  expert  pilots  adapt  to  varying 
conditions  often  without  serious  degradation  in  the 
vehicle's  performance,  but  frequently  with  radical 
changes  in  control  stragety  and  information 
acquisition.  This  and  the  previous  study  have  begun  to 
identify  the  manner  in  which  these  changes  seem  to 
occur . 

The  results  of  this  study,  as  they  concern  only 
experienced  pilots  operating  within  a  particular 
device,  can  be  generalized  only  with  great  caution. 
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1.  Authors:  Isley,  Robert  N. ,  Caro,  Paul  W.  Jr.,  & 

Jolley,  Oran  B. 

2.  Title:  Evaluation  of  Synthetic  Instrument  Flight 
Training  in  the  Officer/Warrant  Officer  Rotary  Wing  Aviator 
Course . 

3.  Source:  Human  Resources  Research  Office  Technical 
Report  68-14,  November  1968 

4.  Topic  Keywords:  Instrument  Flight  Simulation  ; 

Training  Effectiveness  ;  Transfer  of  Training  . 

5.  Short  Summary:  This  study  used  three  groups  of  warrant 
officer  candidates  to  investigate  the  effect  of  differing 
amounts  of  training  time  on  the  altered  flight  training. 

The  results  demonstrated  that  the  different  amounts  of 
training  time  on  the  1-CA-l  rotary  wing  flight  simulator  did 
not  significantly  alter  instrument  flight  proficiency  in  the 
aircraft.  Nothing  else  in  the  training  program  was  altered, 
and  the  fifty  hours  of  flight  time  normally  included  may 
account  for  the  lack  of  differences  in  outcome  measures. 

f)  •  6.  Device:  1-CA-l  Fixed  Wing  Instrument  Trainer  modified 

for  rotary  wing  flight  training 

7.  Institutions: 

a.  Sponsor:  Office,  Chief  of  Research  and 
Development,  Department  of  the  Army,  Washington,  DC 
20310. 

b.  Performing  Organization: 

George  Washington  University  Human  Resources  Research 
Office,  Division  No.  6,  Fort  Rucker,  AL  36362. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  48  Warrent  Officer  Candidates  with 
introductory  helicopter  flight  experience 

(2)  46  same 

(3)  Controls:  51  same 

c.  Tests  or  Trials/Timing:  Post-Test  (Instrument 
Flight  Checkride) 

d.  Number  of  Different  Types  of  Measures:  6 

e.  Description  of  Measurements  and  Racings:  _ 
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(1)  Objectively  scored  photographically  recorded 
data 

(2)  Checkpilot  checklist 

(3)  Checkpilot  grade  for  flight 

(4)  Attrition 

— 

m 

(5)  Time  to  attain  checkride  proficiency 

(6)  Daily  grades  during  training 

f . 

Experimental  Setting/Training  Context: 

Institutional,  hands-on 

i 

g. 

Statistical  Methods:  Chi-Square;  ANOVA 

h. 

Variables  Being  Manipulated: 

(1)  Training  Device:  1-CA-l  fixed  wing  instrument 
trainer  modified  for  rotary  wing  l light  training 

(2)  Fidelity  Levels: 

A 

(a)  Physical:  unspecified;  presumably 

medium 

(b)  Functional:  medium-low 

« 

(3)  Type  of  Task/Skill  Required:  operations; 
cognitive;  psychomotor;  motor;  perceptual; 
procedural;  whole-  task  (helicopter  instrument 
flight) 

(4)  Task  Difficulty:  High 

Stage  of  Training:  familiarization;  skill; 

t  r  - 

ns  it  ion  ito  instrument  flight) 

• 

Trainee  Sophistication:  intermediate 

\  . 

Incorporation  of  Device  into  P.O.I.;  unspecified, 

lit. 

suitably  lockstep 

1  . 

User  Acceptance  or  Attitude: 

j* 

(1)  Instructors:  not  specified 

(2)  Students:  not  specified 

m . 

Use  of  Instrumental  Features:  not  discussed 

9 .  Abstract : 

< 

a . 

Study  Synopsis:  The  objective  of  the  research  was 

to 

oetermine  the  extent  to  which  skills  acquired  during 

•  . 

synthetic  device  training  in  the  tactical  instrument 
phase  of  the  Officer/Warrant  Officer  Rotary  Wing 

Aviator  Course  enhanced  subsequent  trainee  performance 

g 

in 

a  simulated  tactical  environment.  The  synthetic 

training  device  being  used  was  a  fixed-wing  instrument 
flight  simulator  that  had  been  modified  for  rotary-wing 
training.  This  was  the  first  study  of  its 

• 

effectiveness  in  training  rotary  wing  instrument 
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flight,  although  its  effectiveness  had  been  reported 
for  fixed  wing  training. 

Three  groups,  of  48,  46,  and  51  warrant  officer 
candidates  in  an  intermediate  phase  of  training 
(transition  to  instrument-only  flight)  were  given, 
respectively,  1,  10,  and  20  hours  of  simulator  training 
during  an  otherwise  equal  and  standard  course  of 
instruction.  The  20-hour  group  was  the  control  group 
in  that  the  20  simulator  training  hours  were  a  standard 
part  of  the  curriculum.  All  candidates  in  the 
experiment  also  received  50  hours  of  real-aircraft 
inflight  training  during  this  phase. 

All  candidates  were  subsequently  tested  in  a  post- 
experiment  instrument  checkflight  in  the  aircraft,  and 
their  proficiency  rated  according  to  several  measures: 
(1)  objectively  scored  photographic  data  from  the 
checkflight;  (2)  checkpilot  checklist  for  each 
maneuver;  (3)  checkpilot  subjective  grade  for  flight; 
(4)  attrition;  (5)  time  to  attain  checkflight 
proficiency;  (6)  daily  grades  during  training. 

b.  Results: 

(1)  Key  Data:  Mission  error  rates  and  checkride 
grades.  Neither  of  the  ANOVAs  performed  on  these 
data  yeilded  significant  F  ratios. 

Mean  Overall  Mission  Error  Rates 


Mean 

N 

SD 

0-hour  group 

21 

21 

7 

10-hour  group 

22 

19 

9 

20-hour  group 

23 

20 

8 

Total 

22 

60 

8 

Mean  Checkride 

Grades,  by 

Mean 

N 

SD 

0-hour  group 

79.25 

44 

7.68 

10-hour  group 

79.80 

41 

8.37 

20-hour  group 

79.22 

50 

7.54 

(2)  Verbal  Description:  Out  of  96  flight 
measurements,  in  control  and  procedural  instrument 
tasks,  statistically  significant  differences  among 
the  experimental  and  control  groups  occurred  on 
only  three  of  these  measures. 

On  other  measurements  of  proficiency,  no 
significant  differences  were  to  be  found  among  the 
three  groups. 

c.  Authors'  Conslusions:  The  only  reasonable 
conclusion  from  this  research  was  that  no  reliable 
evidence  was  found  to  support  the  assumption  that 
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synthetic  device  training  administered  in  the  modified 
1-CA-l  trainer  during  the  curriculum  improves 
subsequent  aviator  performance  in  a  tactical  instrument 
situation. 

It  should  be  emphasized  that  the  research  described  was 
a  determination  of  the  transfer  of  training  value  of  a 
synthetic  training  program  rather  than  of  a  synthetic 
training  device.  It  is  possible,  for  example,  that  if 
the  present  device  were  used  differently...  some 
evidence  of  beneficial  transfer  might  be  found. 
Preliminary  results  suggest  that  the  synthetic  task,  as 
presently  structured,  bears  little  psychological 
resemblance  to  the  criterion  task.  The  physical 
limitations  of  the  device  are  such  that  little  or  no 
gain  in  transfer  of  skills  to  the  criterion  situation 
can  be  expected  to  result  from  modifications  to  the 
present  synthetic  training  program. 


1.  Author:  Jacobs,  Robert  S. 

2.  Title:  Simulator  Motion  as  a  Factor  in  Flight  Simulator 
Training  Effectiveness. 

3.  Source:  U.S.  Department  Health  Education  &  Welfare 
Education  Resources  Information  Center  Report,  August  1975 

4.  Topic  Keywords:  Flight  Training  Simulation  ; 

Flight  Simulation  ;  Transfer  of  Training  ;  Motion  Cues  ; 
CostEf f ectiveness  . 

5.  Short  Summary:  This  paper  presents  a  review  of 
literature  on  the  effectiveness  of  motion  cues  in  flight 
simulation  and  then  presents  a  description  of  an  experiment 
in  progress  that  is  aimed  at  providing  more  accurate 
knowledge . 

6 .  Devices : 

a.  Piper  PA28R-100  Arrow  single  engine  light  aircraft 

b.  Singer  Link  GAT-2  Training  Simulator;  used  in  3 
conditions : 

(1)  no  motion 

(2)  high  fidelity  washout  motion 

(3)  directionally  uncorrelated  random-roll  washout 
motion 

7.  Institutions: 

a.  Sponsor:  University  of  Illinois  Aviation  Research 
Laboratory,  Urbana,  IL  61801. 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment  (as  described;  has  not 
been  implemented  at  time  of  report). 

a.  Number  of  Groups:  4 

b.  Description  of  Groups:  ( 1 ) — ( 3 )  Subjects:  each 

group,  12  volunteer  student  pilots  without  flying 
experience;  1  group  per  simulator  condition.  (4) 
Controls:  12  same,  no  simulator  training. 

c.  Tests  or  Trials/Timing:  In-process 

tr ials-- tr ials-to-  criterion  are  a  dependent  variable; 
criterion  prof iciency  will  be  an  instructor  pilot 
j udgment . 

d.  Number  of  different  types  of  measures:  2 


e.  Description  of  measurements  and  ratings: 

(1)  Number  of  trials  to  criterion 

(2)  Instructor  Ratings 

f.  Experimental  setting/training  context: 
institutional,  hands-on 


g.  Statistical  methods:  ANOVA 


h.  Variables  being  manipulated: 

(1)  Training  Devices:  as  above 

(2)  Fidelity  Levels: 


(a) 

No  motion 

High-fidelity  washout 
Random-roll  washout 


Physical 

high 

high 

high 


(b)  Functional 
medium-high 
high 
medium-high 


(3)  Type  of  task/skill  required:  operations, 
cognitive,  psychomotor,  perceptual,  whole-task 

(4)  Task  difficulty:  Medium-high 


i.  Stage  of  Training:  introduction,  skill 

k.  Incorporation  of  device  into  P.O.I.: 
instructor- managed 


l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  persumably  to  be  intensive 

(2)  Features  used:  not  specified 


9.  Abstract: 


a.  Study  Synopsis:  This  study  consists  of  a  review  of 
literature  dealing  with  motion  cues  in  flight 
simulation,  and  a  description  of  an  experiment  which 
was  to  be  conducted  at  the  University  of  Illinois  in 
hopes  of  clarifying  the  role  of  motion  cues  in 
simulated  flight  learning. 

Two  principal  issues  are  discussed:  (1)  whether  motion 
cues  in  simulation  contribute  to  transferable1  learning 
m  a  cost-effective  way;  (2)  if  motion  cues  are 
valuable,  what  are  the  kinds  of  cues  that  benefit  the 
student  in  transfer  to  the  aircraft? 


In  regard  to  the  former  issue,  a  number  of  transfer  of 
training  studies  have  failed  to  support  the  notion  that 
motion  in  simulator  training  enhances  the  student's 
later  performance  in  the  aircraft  in  a  cost-effective 
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way,  at  least  after  initial  contact  trials  have  been 
completed. 

Evidence  of  students'  performance  in  the  simulators, 
however,  gives  a  simulator  performance  advantage  to 
students  who  train  with  motion  in  the  practice  task. 

In  one  study  (Koonce  1974),  the  difficulty  of  the  task 
for  the  motionless  simulator  group  apparently  served  to 
strengthen  skills  required  for  superior  aircraft 
performance,  to  the  detriment  of  simulator  performance. 
Therefore,  good  simulator  practice  performance  as  a 
result  of  motion  cues  may  transfer  negatively  to  the 
aircraft. 

With  respect  to  what  motion  cues,  if  any,  might 
contribute  to  transferable  learning,  a  distinction  is 
made  between  cues  whose  intrinsic  characteristics  serve 
the  pilot  as  information,  and  cues  which  serve  merely 
an  alerting  function — that  is,  during  instrument 
flight,  they  merely  prompt  the  pilot  to  check  the 
instruments  from  whose  indications  he  makes  his 
decisions.  The  pilot  knows  that  motion  cues  in  real 
aircraft  instrument  flight  can  be  misleading. 

The  study  planned  to  investigate  these  two  issues  at 
the  University  of  Illinois'  Aviation  Research 
Laboratory  bj  training  four  groups  separately  in  the 
following'  conditions:  ;  i )  simulation  with  no  motion: 

(2)  simulation  with  high  fidelity  washout  motion;  (3) 
simulation  with  directionally  uncorrelated  random-roll 
washout  motion  (in  which  case  motion  would  serve  only 
the  alerting  function) ;  (4)  the  control  condition, 

real  single- engine  light  aircraft  training.  Transfer 
would  be  tested  by  students'  subsequent  performance  in 
aircraft  in  the  next,  pre-solo  phase  of  training.  In 
this  latter  phase,  measurements  were  to  be  the  number 
of  trials  required  to  reach  solo  flight  proficiency,  as 
judged  by  the  instructor  pilot,  and  the  number  of 
errors  to  reach  that  proficiency. 

It  was  expected  that,  if  simulated  motion  did 
contribute  effectively  to  training,  that  significant 
differences  would  be  found  between  the  motion  groups 
and  the  no-notion  group;  and  that,  if  the  real  value 
of  motion  cues  lay  in  the  alerting  function,  that  the 
uncorrelatea  random  roll  motion  group  would  perform  at 
least  as  well  as  the  other . 

b.  Results:  none  at  the  time  of  this  paper's 
compos l t i on 
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c.  Author’s  Conclusions:  It  is  anticipated  that  this 
study  will  reveal  that  the  provision  of  high  fidelity 
roll  motion  simulation  may  facilitate  the  learning  of 
simple  motor  control  and  tracking  to  some  extent,  but 
if  Koonce's  (1974)  findings  are  general i zabl e  to  the 
instrument  referenced  flight  situation,  high  fidelity 
may  hinder  learning  as  compared  with  no  motion  or 
directionally  random  motion  conditions.  With  respect 
to  the  development  of  the  imaginational  components  of 
piloting  skill,  it  is  difficult  to  predict  whether 
motion  characteristics  will  have  any  transfer 
implications  beyond  those  associated  with  heightened 
realism  and  possibly  resultant  increased  motivation  of 
the  student. 


1.  Authors:  Jacobs,  Robert  S.,  Williges,  Robert  C. ,  & 
Roscoe,  Stanley  N. 

2.  Title:  Simulator  Motion  as  a  Factor  in  Flight-Director 
Display  Evalualtion. 

3.  Source:  Human  Factors  ,  1973,  15(6),  569-582. 

4.  Topic  Keywords:  Flight  Simulation  ;  Visual  Displays  ; 
Motion  Perception  ;  Fidelity  ;  Flight  Director  Displays  . 

5.  Short  Summary:  This  experiment  conducted  in  a  high- 
fidelity  flight  simulator  capable  of  motion  shows  that 
motion  cues  in  simulator  learning  can  have  an  important 
impact  on  performance  of  experimental  flight-director 
simulated  tasks.  This  argues  for  the  careful  interpretation 
of  the  results  of  research  done  in  ground-based  simulators, 
considering  the  potential  effects  of  differences  between 
simulated  and  actual  flight  environments. 

6.  Devices:  Link  GAT-2,  high-fidelity  simulator  of  light 
twin-  engine  instrument  flight,  having  flight  controls, 
flight  response  dynamics,  and  instrument  indications 

7.  Institutions: 

a.  Sponsor:  Engineering  Psychology  Programs, 

Office  of  Naval  Research  ,  jointly  with  Life  Sciences 
Program,  Air  Force  Office  of  Scientific  Research 

b.  Performing  Organization:  Aviation  Research 
Laboratory  of  the  Institute  of  Aviation, 

University  of  Illinois  at  Urbana-Champaign . 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  2  (when  results  of  previous 
study  included) 

b.  Description  of  Groups: 

(1)  Subjects:  8  male  pilots  holding  private  pilot 
certificates,  college  students  with  40-150  hours 
flight  experience 

(2)  Controls:  refer  to  Johnson,  S.L.,  Williges, 
R.2.,  and  Roscoe,  S.N.  "A  new  approach  ro  motion 
relations  for  flicht  director  displays"  ARL71- 
20,  ONR-7 1-2  AFGSR-7 1 -6 ,  October  1971 

c.  Tests  or  Trials/Timing:  Single  Post-Test 

d.  Numter  of  Different  Types  of  Measures:  1 
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e.  Description  of  Measurements  and  Ratings: 

Root-mean-  square  error  in  horizontal  steering 
performance,  in  8  3-min.  trials  on  each  of  8  different 
simulated  flight  director  displays 

f.  Experimental  Setting/Training  Context: 
Institutional;  laboratory 

g.  Statistical  Methods:  Two-way  and  Three-way  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  GAT-2  high  fidelity  light 
twin  engine  aircraft  flight  cockpit  simulator 

(2)  Fidelity  Level:  High 

(3)  Type  of  Task/Skill  Required:  Aircraft 
piloting;  psychomotor;  perceptual,  cognitive 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Skill 

j.  Trainee  Sophistication:  Intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  Unspecified; 
presumable  self-paced 

l.  User  Acceptance  or  Attitude:  Not  discussed 

m.  Use  of  Instructional  Features:  Intensive 
Abstract : 

a.  Study  Synopsis:  High  fidelity  in  simulators  used 
for  research  purposes  may  be  more  important  than  in 
simulators  used  for  training  purposes.  Although  pilots 
use  kinesthetic  senses  in  actual  flight,  the 
contribution  of  simulated  motion  cues  to  understanding, 
performing,  and  learning  in-flight  maneuvers  has  net 
been  examined  parametrically.  Ground-based  flight 
simulators  are  incapable  of  providing  completely 
realistic  motion  cues;  moreover,  for  many  tasks, 
motion  cues  become  noise  rather  than  signals. 

This  study  undertook  to  examine,  in  light  of  the 
foregoing  premises,  what  differences  in  test 
performance  on  simulated  flight  maneuve 's  could  be 
attributed  to  the  presence  or  lack  of  me tion  cues 
during  simulated  flight  practice.  Eight  experiencec 
pilots  were  given  two  practice  sessions  in  the 
high-fidelity  simulator  of  a  light  twin-engine 
aircraft,  and  were  thereafter  tested  on  the  same 
device,  for  eight  different  display  configurations. 

The  three  sessions  were  24  hours  apart.  The  subjects 


of  this  experiment  practiced  and  were  tested  without 
motion  cues.  The  performance  was  compared  with  the 
performance  of  another,  comparable  group,  who  had 
undergone  practice  and  testing  by  the  same  method  in  an 
earlier  experiment,  but  who  had  been  given  motion  cues. 
The  test  performance  of  those  trained  and  tested  with 
motion  cues,  for  flight-director  display  maneuvers,  was 
found  superior  to  that  of  those  who  received  no  motion 
cues.  The  authors  conclude  that  "it  has  been 
demonstrated  that  the  fidelity  of  the  motion  cue 
structure  of  flight  simulators  has  an  effect  upon 
performance  levels  observed  in  a  closed-loop,  man- 
control-display-man  system. 

The  application  of  these  findings  to  training  for,  and 
performance  in,  actual  flight  is  subject  to 
investigation. 

b.  Results: 

(1)  Key  Data:  Analysis  of  Variance  Summary  of  Log 
RMS  Horizontal  Tracking  Error  comparing  attitude 
and  command  mode  presentations  in  the  presence  and 
*  absence  of  simulator  motion 


df 

MS 

Motion 

1 

0.0859 

Subjects 

14 

0.0114 

(2)  Verbal  Description:  Pilot  performance  in 
azimuth  steering  on  the  eight  displays  was  better 
with  simultor  motion  than  without.  When  pilots 
flew  the  pursuit  command  steering  mode,  smaller 
RMS  errors  resulted.  Overall,  tracking 
performance  was  superior  when  the  motion  system 
was  on  as  compared  to  off. 

c.  Authors'  Conclusions:  It  has  been 
demonstrated  that  the  fidelity  cf  the  motion  cue 
structure  of  flight  simulators  has  an  effect  upon 
performance  levels  observed  in  a  closed-loop, 
man-control  -display-man  system. . . .  The  term 
"fidelity"  of  motion  cue  reproduction  is  to  some 
extent  misleading  in  that  it  implies  variation 
along  a  single  dimension.  In  actuality,  what  is 
involved  here  is  a  multi-dimensional  variable.... 
The  issue  of  whether  or  not  results  in  fixed-base 
and  moving- base  simulators  can  be  extrapolated  to 
actual  flight  needs  to  be  investigated. 
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1.  Author:  Johnson,  Steven  L. 

2.  Title:  Retention  and  Transfer  of  Training  on  a 
Procedural  Task;  Interaction  of  Training  Strategy  and 
Cognitive  Style 

3.  Source:  Air  Force  Office  of  Scientific  Research  TR 
78-1161,  January  30,  1978. 

4.  Topic  Keywords:  Retention  of  Training  ; 

Transfer  of  Training  ;  Cognitive  Style  ;  Mental  Imagery  ; 
Visual  Imagery  ;  Fidelity  ;  Low  Fidelity  . 

5.  Short  Summary:  This  study  of  a  low  fidelity  training 
media  which  required  trainees  to  use  mental  imagery  found 
that  high  fidelity  is  not  necessary  in  teaching  this 
procedural  task,  and  that  vividness  of  imagery  interacts 
with  training  strategy. 

6.  Devices: 

a.  Actual  Equipment: 

Conveyor  Line  Production  Control  Panel 

W' 

b.  35%  size  Photo  of  Actual  Equipment,  and  pencil 

c.  35%  size  Photo  of  Actual  Equipment,  with 
non-writing  stylus 

7.  Institutions: 

a.  Sponsor:  Life  Sciences  Directorate,  Air  Force 
Office  of  Scientific  Research. 

b.  Performing  Organization:  CALSPAN  Corporation, 
Buffalo,  NY  14221. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  20  age  16-34  paid  male  and  female 
volunteers  without  extensive  previous  experience 
in  procedural  tasks — "Conventional  Strategy"  group 

(2)  18  same  -  "Reproduction  Strategy"  group 

(3)  16  same  -  "Blind  Strategy"  group 

c.  Tests  or  Trials/Timing: 

(1)  "Proficiency  confirmation  trial" 

(2)  Delayed  retention  post-test,  after  70  plus  or 
minus  10  days 


(3)  Transfer  test  following  delayed  retention 
post-test 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Number  of  trials  to  criterion 

(2)  Number  of  errors 

(3)  Time  to  complete  trials 

f.  Experimental  Setting/Training  Context: 
setting/training  context:  laboratory,  hands-on 

g.  Statistical  Methods:  analysis  of  variance  and 
covariance;  regression  analysis 


h. 


Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 

(a)  Actual  equipment,  used  by  "Coventional 
Strategy"  group 

(b)  Photo  and  pencil  used  by  "Reproduction" 
group 

(c)  Photo  and  nonwriting  stylus,  used  by 
"Blind  Strategy"  group 

(2)  Fidelity  Levels: 


Physical 

Actual  Equipment  High 

Reproduction  Low 

Blind  Low 


Functional 
not  specified 
Low 
Low 


(3)  Type  of  Task/Skill  Required:  set-up  conveyor 
line;  operations;  procedural;  whole-task 

(4)  Task  Difficulty:  medium 


i.  Stage  of  Training:  introduction,  experimental 


j.  Trainee  Sophistication:  novice-intermediate 


k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 


l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  as  intensive  as  possible 

(2)  Features  used:  virtually  no  simulator 
feedback  features 
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9.  Abstract: 


a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
investigate  the  effectiveness  of  visual  imagery 
techniques  as  they  pertain  to  initial  training, 
retention,  and  transfer  of  training.  In  addition  to 
investigating  the  relative  merit  of  using  visual 
imagery  versus  conventional  training  methods,  the 
possibility  of  matching  instructional  strategy  and  the 
trainee's  cognitive  style  was  investigated. 

The  task  chosen  for  study  was  the  operation  of  a 
control  panel  to  set  up  a  conveyor  line  production  job, 
which  required  proformance  of  a  sequence  of  87  discrete 
simple  actions.  A  subsequent  "transfer"  task  consisted 
of  83  sequential  actions. 

The  three  training  modes  were  (1)  operating  the  actual 
equipment;  (2)  writing  actions  on  a  35%  size 
photographic  reproduction  of  the  equipment;  and  (3) 
imagining  actions  on  the  photograph,  without  being  able 
to  write  on  it  (although  actions  could  be  simulated 
with  a  nonwriting  stylus) .  54  paid  male  and  female 

volunteers,  age  16-34,  without  extensive  prior 
experience  in  procedural  tasks,  were  divided  into  3 
groups,  one  for  each  training  medium.  The  two  groups 
trained  on  the  low  fidelity  devices  were  briefly 
familiarized  with  the  actual  equipment  before  training 
began.  All  subjects  were  given  two  visual  imagery 
aptitude  tests  in  order  to  assess  the  interaction 
between  learning  and  visual  imagery  ability. 

After  completion  of  training  to  a  prescribed  criterion, 
each  trainee  performed  a  "proficiency  confirmation 
trial"  on  the  operational,  actual  equipment.  After  a 
delay  of  from  60  to  80  days,  trainees  were  given  two 
retention  trials,  and  a  relearning  session  to 
criterion.  They  were  also  given  a  "transfer"  trial  on 
a  similar  task.  NOTE:  this  "transfer"  trial  differs 
from  the  usual  sense  of  transfer  in  that  it  involved  a 
different  task  on  the  same  equipment.  In  fact,  the 
simulator-trained  groups  were  given  a  transfer  test,  in 
the  more  common  sense,  when  they  operated  the  actual 
equipment  in  the  proficiency  trial. 

Performance  measures  were:  (1)  number  of  trials  to 
criterion;  (2)  number  of  errors  in  the  performance  of 
the  task;  and  (3)  time  to  complete  trials. 
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Results : 

(1)  Key  Data: 

Strategy 

Measure  Conventional  Production  Blind 


r  Beta 

Initial  Training 

Trials  to  Criterion 

r 

Beta 

r 

Beta 

.38  .045** 

Total  errors 

.09 

.016 

.47 

.103** 

.43  .802** 

Total  Training  Time 

.25 

.538 

.45 

1.277** 

.45  .  .861** 

Retention  Evaluation 
Total  Errors 

.22 

.572 

.62 

2.870** 

.74  .496** 

Decrease  in  Errors 

.12 

.040 

.22 

.144 

.56  .265** 

Retraining 

Trials  to  Criterion 

.29 

.083 

.11 

.065 

.48  .052** 

Total  Errors 

.18 

.012 

.41 

.035 

.54  .424** 

Total  Training  Time 

.22 

.129 

.46 

.540** 

.51  .464** 

Transfer  of  Training 
Trials  to  Criterion 

.18 

.095 

.39 

.393 

.09  .009 

Total  Errors 

.30 

.038 

.39 

.076 

.02  .017 

Total  Training  Time 

.29 

.461 

.41 

1.047 

.22  .158 

.33 

.443 

.35 

.740 

**  Indicates  significant 
level. 

Beta 

value 

at  p< 

.05 

(2)  Verbal  Description:  The  strongest  effect 
during  original  training  was  the  difference  among 
the  three  groups'  training  times,  indicated  for 
total  training  time,  time  to  perform  the  first 
trial,  and  time  to  perform  the  last  trial.  During 
the  first  trial,  the  conventionally  trained  group 
performed  fastest,  the  reproduction  group  next 
fastest,  the  "blind"  group  slowest.  However,  only 
the  "blind"  group  differed  significantly  in  total 
training  time  and  time  to  perform  the  last  trial. 
The  results  of  this  study  indicate  that  the  total 
time  to  learn  on  the  blind  strategy  is 
approximately  1.5  times  as  long  as  that  required 
to  learn  on  the  conventional  strategy. 


There  were  no  significant  differences  among  the 
three  groups  in  trials  to  criterion,  or  total 
number  of  errors  during  training,  as  shown  in  the 
table. 

On  the  retention  trials,  the  reproduction  strategy 
was  significantly  superior  to  the  conventional 
strategy  on  the  basis  of  the  total  number  of 
errors,  with  -he  blind  strategy  being  between  the 
other  two  (not  significantly  different  from 
either) . 

Refresher  training  results  indicated  that  training 
device  fidelity  does  not  have  to  be  high  for 
refresher  training. 

In  both  training  and  transfer  portions  of  the 
experiment,  no  differences  were  significant  in 
terms  of  transfer  from  the  experimental  devices  to 
the  operational  equipment  (p<.25).  In  the 
retraining  portion  of  the  study,  the  differences 
were  not  significant  (although  they  were  not  above 
the  .25  level) . 

The  individual  capability  for  vividness  of  imagery 
had  a  much  more  predominant  effect  within  the 
conventional  training  strategy  than  within  the 
other  modes.  Referring  to  the  table  above,  15  of 
the  26  relationships  between  imagery  and  the 
performance  measures  were  statistically 
significant  when  using  the  conventional  strategy. 
For  the  reproduction  strategy,  only  one 
relationship  was  significant,  while  there  were 
eight  for  the  blind  strategy.  The  results 
indicate  that  the  conventional  strategy  is 
sufficient  or  possibly  superior  for  the  less  vivid 
imager  but  that  it  handicaps  the  more  vivid 
imager.  Performance  of  low  imagers  was  comparable 
to  the  performance  of  high  imagers  when  the 
reproduction  strategy  was  used. 

An  analysis  of  sex  differences  revealed  that  the 
females  tend  to  be  able  to  transfer  from  one  task 
to  another  better  than  males. 

c.  Author's  Conclusions:  Vividness  of  imagery 
does  interact  with  training  strategy.  Performance 
can  be  enhanced  by  matching  the  training  strategy 
with  the  trainee's  cognitive  style. 


The  experimental  strategies  were  found  to  have 
advantages  for  retention  apparently  without 
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accompanying  problems  during  training.  These 
results,  along  with  the  cost  benefits  of  using  the 
simpler  types  of  devices  make  the  use  of  such 
training  methods  desirable.  Training  devices  do 
not  need  to  be  high  fidelity  for  procedural  tasks. 
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1.  Author:  Johnson,  Steven  L. 

2.  Title:  Effect  of  Training  Device  on  Retention  and 
Transfer  of  a  Procedural  Task. 

3.  Source:  Human  Factors  ,  1981,  23(3),  257-272. 

4.  Topic  Keywords:  Simulation  Fidelity  ; 

Transfer  of  Training  ;  Retention  of  Training  ; 

Procedural  Task  ;  Visual  Imagery  . 

5.  Short  Summary:  This  study  compared  three  training 
devices  of  varying  fidelity  in  training  the  same  task.  The 
conclusion  was  that:  (1)  training  devices  do  not  need  to  be 
of  high  fidelity  to  be  effective  in  training  procedural 
tasks,  and  (2)  a  lower-fidelity  device  may  provide  superior 
retention  in  a  procedural  task. 

6.  Devices: 

a.  Actual  Equipment: 

Conveyor  Line  Production  Control  Panel. 

b.  35%  Photo  of  Actual  Equipment,  and  pencil. 

c.  35%  Photo  of  Actual  Equipment,  with  non-writing 
stylus. 

7.  Institutions: 

a.  Sponsor:  Life  Sciences  Directorate, 

Air  Force  Office  of  Scientific  Research. 

b.  Performing  Organization:  CALSPAN  Corporation, 
Buffalo,  NY  14221. 

8.  Type  of  Article:  Experiment. 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  "Conventional  Strategy"  group:  20  age  16-34 
paid  male  and  female  volunteers  without  extensive 
previous  experience  in  procedural  tasks 

(2)  "Reproduction  Strategy"  group:  18  same 

(3)  "Blind  Strategy"  group:  16  same 

c.  Tests  or  Trials/Timing: 

(1)  "Proficiency  confirmation  trial" 

(2)  Delayed  retention  post-test,  after  60-80  days 

(3)  Transfer  test  following  delayed  retention 
post-test 


d.  Number  of  Different  Types  of  Measures:  3 


e.  Description  of  Measurements  and  Ratings: 

(1)  Number  of  trials  to  criterion 

(2)  Number  of  errors 

(3)  Time  to  complete  trials 

f.  Experimental  Setting/Training  Context:  laboratory 
hands-on 

g.  Statistical  Methods:  analysis  of  variance  and 
covariance;  regression  analysis 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Actual  equipment:  conveyor  line 
production  control  panel 

(b)  35%  photo  of  acutal  equipment  and  pencil 

(c)  35%  photo  of  actual  equipment,  with 
non-writing  stylus 

(2)  Fidelity  Levels: 

Physical  Functional 
Actual  equip.  Very  High  not  specified 
Reproduction  Low  Low 

Blind  Low  Very  Low 

(3)  Type  of  Task/Skill  Required:  operations; 
procedural;  whole-task 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  introduction,  experimental 

j.  Trainee  Sophistication:  novice-intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  low 

(2)  Features  used:  virtually  no  simulator 
feedback  features 


Abstract : 


a.  Study  Synopsis:  This  study  investigated  three 
different  aspects  of  training  device  effectiveness. 
Training  device  fidelity  was  investigated  by  using 
three  different  devices  which  required  different 
degrees  of  visual  imagery.  The  effect  of  training 
device  on  retention  of  the  skill  and  on  ability  to 
transfer  to  a  different  procedure  were  also 
investigated. 

The  task  chosen  for  study  was  the  operation  of  a 
control  panel  to  set  up  a  conveyor  line  production  job, 
which  required  performance  of  a  sequence  of  87  discrete 
simple  actions. 

The  three  training  modes  were:  (1)  operating  the 
actual  equipment;  (2)  writing  actions  on  a  35% 
photographic  reproduction  of  the  equipment;  (3) 
imagining  actions  on  the  photograph,  without  being  able 
to  write  on  it  (although  actions  could  be  simulated 
with  a  non-  writing  stylus) .  These  three  modes  were 
designated  respectively,  the  "Conventional,"  the 
"Reproduction, "  and  the  "Blind. " 

Fifty-four  paid  male  and  female  volunteers  age  16-  34, 
without  extensive  prior  experience  in  procedural  tasks, 
were  divided  into  three  groups,  one  for  each  training 
condition.  The  two  groups  trained  on  the  low  fidelity 
devices  were  briefly  familiarized  with  the  actual 
equipment  before  training  began.  After  completion  of 
training  to  a  prescribed  criterion,  each  trainee 
performed  a  "proficiency  confirmation  trial":  on  the 
operational  actual  equipment.  After  a  delay  of  60-80 
days,  trainees  were  given  two  retention  trials,  and  a 
relearning  session  to  criterion.  They  were  also  given 
a  "transfer"  trial  on  a  similar  task. 

NOTE:  This  "transfer"  trial  differs  from  the  usual 

sense  of  transfer  in  that  it  involved  a  different  task 
on  the  same  equipment  used  for  training  the 
"Conventional"  group.  In  fact,  the  simulator-trained 
groups  were  given  a  transfer  test,  in  the  more  usual 
sense,  when  they  operated  the  actual  equipment  in  the 
"proficiency  confirmation  trial"  at  the  conclusion  of 
initial  training. 

Performance  measures  were:  (1)  number  of  trials  to 
criterion;  (2)  number  of  errors  in  the  performance  of 
the  task;  (3)  time  to  complete  trials. 


b.  Results: 

(1)  Key  Data: 


Strategy 

Table 

Conven. 

Repro 

Blind 

df 

F 

Initial  Training 
Time  71.86 

84.80 

106.71 

(Transformed)  8.35 

9.02 

10.12 

2,56 

4.15 

p<0 .021 

Trials  to 

Criterion  6.55 

7.61 

7.75 

2,56 

1.15 

Total  Errors  49.85 
Retention 

62.17 

73.44 

2,56 

1.68 

Total  Errors  42.05 

37.50 

39.81 

2,50 

3.44 

p<0 .040 

Transfer  Trials 

to  Crit.  4.25 

4.33 

5.31 

2,50 

1.30 

Total  Errors  21.85 

23.72 

37.69 

(Transformed)  4.49 

4.38 

5.79 

2,50 

2.10 

(2)  Verbal  Description:  The  strongest  effect 
during  original  training  was  the  difference  among 
the  three  groups'  training  time.  During  the  first 
trial,  the  conventionally  trained  group  performed 
fastest,  the  reproduction  group  next  fastest,  the 
blind  group  slowest.  However,  only  the  "blind" 
group  differed  significantly  in  total  training 
time  and  time  to  perform  the  last  trial.  The 
results  of  this  study  indicate  that  the  total  time 
to  learn  on  the  blind  strategy  is  approximately 
1.5  times  as  long  as  that  required  to  learn  on  the 
conventional  strategy. 

There  were  no  significant  differences  among  the 
groups  in  trials  to  criterion,  or  total  number  of 
errors  during  training. 

On  the  retention  trials,  the  reproduction  training 
was  significantly  superior  to  the  conventional 
training  on  the  basis  of  the  total  number  of 
errors,  with  the  "blind"  training  falling  between 
the  other  two  (not  significantly  different  from 
either) . 

In  both  training  and  transfer  portions  of  the 
experiment,  no  differences  were  significant  in 
terms  of  transfer  from  the  experimental  devices  to 
the  operational  equipment. 

c.  Author's  Conclusions:  A  training  device  does  not 
need  to  be  of  high  fidelity  for  the  training  of 
procedure-  following  tasks.  In  fact,  a  low-fidelity 
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training  condition  may  lead  to  improved  retention  of 
the  initial  learning. 
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1.  Authors:  Johnson,  William  B. ,  &  Rouse,  William  B. 

2.  Title:  Training  Maintenance  Technicians  for 

Troubleshooting:  Two  Experiments  with  Computer  Simulations 

3.  Source:  Human  Factors  ,  1982,  24(3),  pp.  271-276 

4.  Topic  Keywords:  Aviation  Maintenance  Training  ; 
Computer  Simulations  ;  Troubleshooting  ; 

Context-free  Training  . 

5.  Short  Summary:  A  comparison  of  computer-based 
simulation  training  with  more  traditional  training  for 
troubleshooting  maintenance  tasks  finds  the  computer-based 
training  competitive  in  tr ansf er-of-training  effectiveness, 
and  possibly  less  costly. 

6.  Devices: 

a.  Computer  Simulations  (experimental) 

b.  Video  and  written  materials  (control) 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences 

b.  Performing  Organization:  Aviation  Research 
Laboratory,  University  of  Illinois  ,  Savoy,  IL  61874 

8.  Experiment:  (Two  Experiments) 

a.  Number  of  Groups:  Experiment  1,  3;  Experiment  2, 
2 

b.  Description  of  Groups: 

EXPERIMENT  1 

( 1 ) — ( 2 )  Subjects:  12  each  group,  advanced 
aviation  maintenance  trainees  enrolled  in  final 
course  prior  to  FAA  certification 
(3)  Controls:  12  same 
EXPERIMENT  2 

(1)  Subjects:  11  same  as  Experiment  1 

(2)  Controls:  11  same  as  Experiment  1 

c.  Tests  or  Trials/Timing:  Post-test,  both 
experiments 

d.  Number  of  Different  Types  of  Measures:  3 
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1.  Authors:  Krahenbuhl,  G.  S.  ,  Marett,  J.  R.  ,  & 

Reid,  G.  B. 

2.  Title:  Task-specific  Simulator  Pretraining  and 
In-flight  Stress  of  Student  Pilots 

3.  Source:  Aviation,  Space,  and  Environmental  Medicine, 
1978  September,  49,  1107-1110 

4.  Topic  Keywords:  Task-specific  simulation  ; 

Flight  simulation  . 

5.  Short  Summary:  Analysis  of  aircraft  flight  urine 
samples  from  student  pilots  trained  with  and  without 
simulated  power-on  stalls  and  spin  recoveries  suggests  that 
task-specific  training  in  the  simulator  can  enhance  the 
students'  emotional  adaption  to,  and  work  capability  on, 
subsequent  real-aircraft  flight. 


6.  Devices: 

a.  Advanced  Simulator  for  Undergraduate  Pilot  Training 

b.  T-37  training  jet  aircraft 

7.  Institutions: 


a.  Sponsor:  United  States  Air  Force 

b.  Performing  Organization:  Human  Performance 
Laboratory,  Arizona  State  University  ,  Tempe,  AZ  85281 


8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  10  USAF  T-37  pilot  trainee 
volunteers 

(2)  Controls:  10  same 

c.  Tests  or  Trials/Timing:  1  post-test  (checkf light) 


d.  Number  of  different  types  of  measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Urinalysis  for  catecholamines  (preflight  and 
postflight) 

(2)  Performance  Score 


f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 
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reliable  than  conventional  subjective  ratings. 

The  low  correlation  of  pilot  self-evaluations  with 
scored  performance  indicates  low  criterion-related 
validity. 
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(2)  Verbal  Description: 


(a)  Performance:  only  the  no-motion  group 
showed  significant  improvement  from  Day  1  to 
Day  2  and  from  Day  2  to  Day  3  (simulator  to 
aircraft) .  For  all  3  groups.  Day  3 
performance  in  the  aircraft  was  significantly 
better  than  Day  1  performance  in  the 
simulator.  By  Day  3,  there  were  no 
significant  differences  between  the  three 
gorups  for  either  IFR  or  VFR  maneuvers  in  the 
aircraft. 

(b)  Prediction  and  Validation  of  Scoring 
device:  the  multiple  correlations  from  Day  2 
(GAT)  to  Day  3  (aircraft)  for  Groups  I,  II, 
and  III,  were  0.722,  0.874  (p<0.01),  and 
0.647  using  the  total  mission  scores  as  the 
criteria,  and  0.763  (p<0.05)  0.911  (p<0.01), 
and  0.651  using  the  observers'  overall 
subjective  flight  mission  rating  as  the 
criteria.  The  correlations  between  the 
criteria  of  the  total  mission  scores  and 
observer  overall  mission  ratings  were  0.875, 
0.852,  and  0.726  for  the  three  groups  on  Day 

•  3  (p<0.01). 

(c)  Pilots'  self-evaluation:  the 
correlations  between  the  subjects'  confidence 
ratings  and  their  subsequent  performance  of 
the  maneuvers  were  not  significantly  greater 
than  zero  and  averaged  0.262  in  the  simulator 
and  0.224  in  the  aircraft. 

c.  Author's  Conclusions:  The  data  of  this  study  tend 
to  support  the  belief  that  the  no  motion  group  would 
perform  better  in  the  aircraft  on  the  contact 
maneuvers.  However,  they  were  not  different  from  the 
other  groups  in  the  aircraft  on  the  highly  practiced 
instrument  maneuvers,  for  the  reason,  apparently,  that 
they  did  not  develop  dependence  on  motion  cues  in  the 
simulator . 

The  results  indicated  that  the  sustained  motion  system 
resulted  in  the  best  prediction  of  aircraft  flight 
proficiency. 

The  Pilot  Performance  Record  scores'  observer-to- 
observer  correlation  and  its  correlations  with  standard 
overall  observer-rated  mission  scores  suggest  the 
feasibility  of  establishing  a  performance  scoring 
device  that  is  perhaps  more  quantifiable,  detailed,  and 
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their  ability  with  their  measured  performance,  to 
assess  the  validity  of  pilot's  self-evaluation 

90  licensed  multi-engine  pilots  were  divided  into  three 
groups  matched  according  to  experience.  Each  group 
flew  two  trials  of  ten  maneuvers  (5  instrument,  5 
visual)  on  the  simulator  under  one  of  three  motion 
conditions:  no-motion;  sustained  linear  scaled-down 

analog  motion  standard  on  the  GAT-2;  and  a  modified 
"wash-out"  motion  that  returned  the  simulator  cab 
toward  a  near  threshhold  rate  following  the  application 
of  an  acceleration.  Subsequently  all  subjects  flew  the 
same  10  maneuvers  in  a  Piper  Aztec  light  twin-engine 
aircraft.  10  subjects  from  each  group  flew  still 
another  identical  series  of  maneuvers  in  the  aircraft 
following  the  first  mission. 

Each  simulated  and  real  mission  was  scored  by 
observbers  both  per  maneuver  and  per  overall  mission. 
The  "Pilot  Performance  Record"  developed  by  the  author 
composed  the  per-maneuver  rating,  of  which  the  scales 
were  well-defined,  easy  to  follow,  descriptive  of  the 
maneuvers  and  behaviors  being  recorded,  and.  not  so 
demanding  upon  the  observer  as  to  consistute  a  safety 
hazard.  The  overall  mission  rating  was  a  standard 
observer  subjective  rating. 

Subjects  also  rated  their  confidence  to  perform  each 
maneuver  before  their  flights. 

b.  Results: 

(1)  Key  Data: 

Composite  error  scores  for  each  group  on  five 
contact  maneuvers  and  five  instrument  maneuvers 
( N=30/group) 


Day  1  Day  2  Day  3 

Contact:  Simulator  Simulator  Aircraft 


No  Motion  Group 

8  .0 

7.0 

4.1 

Sustained  Motion  Gp. 

7.1 

5.9 

5.2 

Washout  Motion  Gp. 
Instrument : 

6.2 

5.5 

5.3 

No  Motion  Group 

5.8 

4.6 

3.9 

Sustained  Motion  Gp. 

5.2 

4.0 

4.5 

Washout  Motion  Gp. 

5.0 

3.7 

3.9 

Reviewer's  Note:  The 

foregoing 

f igures 

are  my 

approximations  for  graphic  representation  of  the 
data. 


231 


f.  Experimental  setting/training  context:  Laboratory, 
hands-on 

g.  Statistical  methods:  ANOVA;  ANOVA  SPF-p.q. 
design;  Sheffle  tests;  Pearson  product-moment 
correlations 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  above,  section  6 

GAT-2  flight  simulator,  3  conditions 
(a)  No  motion 

lb)  Sustained  linear  scaled  down  analog 
motion 

(c)  Washout  motion 

(2)  Fidelity  Levels: 

Physical  Functional 
No  Motion  High  Medium 

Sustained  Motion  High  Medium-high 

Washout  Motion  High  Medium-high 

(3)  Type  of  task/skill  requred:  operations; 
cognitive;  psychomotor;  perceptual;  part-task 

(4)  Task  difficulty:  Medium-high 

i.  Stage  of  training:  familiarization,  skill 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  not  specified;  presumed  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  this  study  was 
fivefold: 

(1)  to  determine  if  flying  proficiency  can  be 
predicted  from  performance  in  a  ground-based 
simulator ; 

(2)  if  simulators  yield  predictive  validity,  how 
is  it  influenced  by  simulator  motion  conditions; 

(3)  to  validate  a  pilot  performance  scoring  device 
developed  by  the  author; 

(4)  to  compare  the  aforementioned  scoring  device 
with  traditional  subjective  ratings  of  pilot 
performance ; 

(5)  to  compare  pilots'  level  of  confidence  in 
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1.  Author:  Koonce,  Jefferson  M. 

2.  Title:  Predictive  Validity  of  Flight  Simulators  as  a 
Function  of  Simulator  Motion 

3.  Source:  Human  Factors  ,  1979,  21(2),  215-223 

4.  Topic  Keywords:  Flight  simulation  ;  Motion  cues  ; 
Predictive  validity  ;  Training  effectiveness  . 

5.  Short  Summary:  A  study  of  the  effects  of  levels  of 
motion  on  training  effectiveness  and  the  predictive  validity 
of  flight  simulators  through  a  transfer  of  training  design, 
found  that  simulator  motion  did  not  result  in  better 
aircraft  performance,  and  higher  predictive  validity  was 
found  with  very  basic  sustained  motion. 

6.  Devices: 

a.  Piper  Aztec  light  twin-engine  aircraft 

b.  GAT-2  flight  simulator 

7.  Institutions: 

a.  Sponsor:  Life  Sciences  Program, 

Air  Force  Office  of  Scientific  Research 

b.  Performing  organization: 

University  of  Illinois  at  Urbana-Champaign 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups  :  3 

b.  Description  of  Groups: 

(l)-(3)  Subjects:  each  group,  approximately  30 
volunteers  multi-engine  FAA  licensed  instrument¬ 
rated  pilots,  matched  for  total  multi-engine 
flying  time  and  amount  of  instrument  flying  time 
logged  during  past  6  months 

c.  Tests  or  trials/timing:  1  post-test  (checkf light) ; 
2nd  post  test  (checkf light)  for  1/3  of  participants 

d.  Number  of  different  types  of  measures:  3 

e.  Description  of  measurements  and  ratings: 

(1)  Observers'  ratings  for  each  maneuver  (Pilot 
Performance  Record) 

(2)  Observers'  overall  rating  for  each  mission 

(3)  Student  pilots'  self-confidence  ratings 
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context-free  tasks  to  the  equipment-specific 
simulations.  This  transfer  is  most  pronounced  when 
students  can  work  on  the  problems  in  a  self-paced 
situation  using  computer  aiding.  Results  were  to  some 
extent  dependent  on  the  level  of  experience  of  the 
trainees;  less-experienced  trainees  showed  less 
transfer  from  computer-aided  to  unaided  contexts. 

c.  Authors'  Conclusions:  Transfer  studies  need  to  be 
continued,  to  determine  the  degree  of  transfer  from 
context-free  tasks  and  equipment-specific  simulations 
to  hands-on,  actual  equipment  troubleshooting. 

RESEARCH  EFFORT  3  (Adaptive  Computerized  Training  System) 

a.  Study  Synopsis:  The  Adaptive  Computerized  Training 
System  employs  a  computer-generated  model  of  "expert" 
decision-making  which  is  compared  with  student 
decision-  making  during  the  performance  of  simulated 
troubleshooting  tasks.  The  student  receives  feedback 
from  the  computer  to  help  make  his/her  decisions  more 
like  an  expert's.  This  research  differs  from  those 
previously  discussed  in  that  here  the  context  is  task- 
specific. 

b.  Results:  The  learning  algorithms  in  the  system  do 
learn  to  predict  human  performance.  Student 
performance  improves  with  practice  on  the  system,  even 
when  no  feedback  based  on  the  student  model  is 
provided.  In  simulation  studies,  similar  sets  of 
utilities  produce  similar  troubleshooting  strategies, 
while  dissimilar  sets  of  utilities  produce  dissimilar 
troubleshooting  strategies. 

c.  Authors'  Conclusions:  Further  evaluation  of 
training  and  cost  effectiveness  of  the  ACTS  are  to  be 
made. 


logic  circuit  diagrams  at  a  lower  level  of  complexity 
than  with  the  first  group  (Logic  Game  Group) ;  (3) 

chance  game-playing  followed  by  reading  logic  circuit 
diagrams  at  the  same  level  of  complexity  as  the  second 
group  (Control  Game  Group) .  Groups  were  then  tested 
for  transfer  by  attempting  to  solve  logic  circuit 
diagrams  at  various  levels  of  complexity  including  the 
highest. 

b.  Results:  Two  general  trends  were  evident  in 
performance  in  the  transfer  test.  The  Logic  Control 
Group  was  superior  to  the  Game  Control  Group.  The 
Logic  Game  Group  was  in  the  middle,  significantly  worse 
than  the  Logic  Control  Group  in  terms  of  accuracy 
scores,  but  no  performance  scores;  and  significantly 
better  than  the  Game  Control  Group  in  terms  of 
performance  scores,  but  not  accuracy  scores.  The 
second  trend  is  that  the  Logic  Game  Group  performed 
more  like  the  Game  Control  Group  on  the  diagrams  of 
intermediate  complexity,  and  more  like  the  Logic 
Control  Group  on  diagrams  of  high  complexity. 

c.  Authors'  Conclusions:  Under  certain  conditions, 
limited  practice  in  reading  logic  circuit  diagrams, 
when  combined  with  playing  a  logic  game  like  the  one 
used  here,  is  as  effective  as  the  same  amount  of  time 
spent  in  practicing  reading  a  more  extensive  set  of 
diagrams.  This  is  not  true  when  the  limited  practice 
is  combined  with  playing  a  game  of  chance,  so  results 
cannot  be  attributed  to  any  general  transfer  effects  of 
game  playing  per  se.  Because  both  game  groups  had  the 
same  practice  set  of  diagrams,  it  also  cannot  be  argued 
that  limited  practice  was  sufficient  for  successful 
performance  on  the  transfer  test. 

RESEARCH  EFFORT  2  (Fault  Diagnosis:  Context-free  Trouble¬ 
shooting) 

a.  Study  Synopsis:  Research  in  the  area  of  "Human 
Performance  in  Fault  Diagnosis  Tasks"  hypothesizes 
skills  common  to  all  fault  diagnostic  tasks. 

Experiments  in  this  area  investigated  the  transfer  of 
training  from  one  context-free  task  to  another,  and 
from  context-free  tasks  to  equipment-specific  tasks. 

Five  experiments  were  performed,  using  engineering 
students  and  aviation  mechanic  trainees  in  first  and  in 
fourth  semesters. 

b.  Results:  Some  positive  transfer  was  observed. 
Positive  transfer  did  take  place  in  some  cases  between 
the  two  levels  of  context-free  tasks,  and  from  the 
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e.  Description  of  Measurements  and  Ratings:  Cost, 
based  on 

(1)  Time  to  complete  action 

(2)  Cost  of  replacement  items 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Computerized  context-free 
troubleshooting  simulator 

(2)  Fidelity  Levels: 

(a)  Physical:  Low 

(b)  Functional:  Medium-high 

(3)  Type  of  Task/Skill  Required:  cognitive, 
psychomotor,  perceptual 

(4)  Task  Difficulty:  Medium  to  medium-high 

i.  Stage  of  Training:  experimental 

j.  Trainee  Sophistication:  novice  in  experiments  1 

«•  and  2;  beginning  in  experiment  4; 

intermediate-advanced  in  experiments  3  and  5 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

9.  Abstract:  This  preliminary  report  describes  three 
research  efforts  in  progress  in  the  area  of  computer-based 
maintenance  simulation.  The  advantages  of  computer-based 
simulations  are  in  cost,  versatility  of  simulation,  range  of 
available  instructional  strategies,  and  adaptability  to 
individual  student  differences.  This  study  is  aimed  at 
developing  ways  to  train  generalized  student  skills  that  can 
be  applied  to  variety  of  items  of  equipment. 

RESEARCH  EFFORT  1  (Game-based  Learning) 

a.  Study  Synopsis:  The  first  research  effort,  labeled 
"Game-based  Learning, "  investigated  the  effectiveness 
of  computer  game-playing  in  teaching  the  reading  of 
logic  circuit  diagrams.  Three  groups  received  training 
via  (1)  conventional  instruction  in  logical  functions 
and  reading  logic  circuit  diagrams  (Logic  Control 
Group) ;  (2)  logic  game-playing  followed  by  reading 
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(2)  "Performance"  (quantity  of  work  done 
correctly) 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Computer  games 

(b)  Computerized  context-free  troubleshooting 
simulator 

(c)  Adaptive  Computerized  Training  System 
employing  artificial  intelligence 

(2)  Fidelity  Levels: 

(a)  Physical:  Low 

(b)  Functional:  Low  to  medium-low 

(3)  Type  of  Task/Skill  Required:  cognitive, 
psychomotor 

(4)  Task  Difficulty:  medium-high 

i.  Stage  of  Training:  experimental 

j.  Trainee  Sophistication:  experimental 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

RESEARCH  EFFORT  2:  FAULT  DIAGNOSIS  (CONTEXT-FREE 
TROUBLESHOOTING) 

a.  Number  of  Groups:  15 

b.  Description  of  Groups: 

(1)  Experiment  One:  3  groups  engineering  students 

(2)  Experiment  Two:  3  groups  same 

(3)  Experiment  Three:  3  groups  Aviation  mechanics 
trainees,  4th  semester 

(4)  Experiment  Four:  3  groups  same,  1st  semester 

(5)  Experiment  Five:  3  groups  same,  4th  semester 

c.  Tests  or  Trials/Timing:  post-tests 


d.  Number  of  Different  Types  of  Measures:  1 


1.  Authors:  Knerr,  Bruce  W. ,  Simutis,  Zita  M. ,  & 

Johnson,  Richard  M. 

2.  Title:  Simulation  Approaches  to  Maintenance  Training 

3.  Source:  U.S.  Army  Research  Institute  Technical  Report 
544,  1980 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Game-based  Learning  ;  Computer-Assisted  Instruction  ; 
Maintenance  Training  ;  Instructional  Games  ; 

Context-free  Fault  Diagnostic  Tasks  ; 

Artificial  Intelligence  . 

5.  Short  Summary:  Research  efforts  underway  to  develop 
ways  of  training  generalized  student  maintenance  skills 
through  computer-based  simulation  suggest  promising  paths  of 
investigation. 

6.  Devices: 

a.  Computer  games 

^) •  b.  Computerized  context-free  troubleshooting  simulator 

c.  Adaptive  Computerized  Training  System  employing 
artificial  intelligence 

7.  Institutions: 

a.  Sponsor:  U.S.  Army 

b.  Performing  Organization:  U.S.  Army  Research 
Institute  for  the  Behavioral  and  Social  Sciences,  5001 
Eisenhower  Ave.,  Alexandria,  VA  22333 

8.  Type  of  Article:  Experiment  (two  sets  of  experiments) 

RESEARCH  EFFORT  1 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

( 1 ) — ( 2 )  Subjects:  each  14  Army  enlisted  personnel 
(3)  Controls:  same 

c.  Tests  or  Trials/Timing:  post-test 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Accuracy 
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(2)  Verbal  Description: 

(a)  Experiment  1:  "The  Video  trainees  had 
significantly  higher  average  performance 
index  than  those  trained  with  either  of  the 
computer  simulations,  F(2r33)  ■  6.27,  p  < 
0.01." 

"The  Video  trainees  were  not  significantly 
faster  than  the  TASK  and  FAULT  trainees; 
however,  the  Video  trainees  did  have  a 
significantly  higher  evaluator's  rating,  F 
(2,33)  =  4.92,  p  <  0.025." 

"Performances  with  the  simulations  were  good 
enough  to  equal  those  with  Video,  as  long  as 
explicit  solution  sequences  were  not 
presented  in  Video." 

(b)  Experiment  2:  "There  were  no  significant 
differences  between  training  methods  for  any 
of  the  three  performance  measures:  average 
performance  index,  F(l,20) *0.89;  average 
time,  F(l,20)  *  1.31;  and  average 
evaluator's  rating,  F(l,20) =0.09.  TASK/FAULT 
did  improve  relative  to  Video  in  Experiment 
2." 

c.  Authors'  Conclusions:  An  appropriate  combination 
of  low  and  moderate  fidelity  computer  simulations  can 
provide  sufficient  problem-solving  experience  to  be 
competitive  with  the  more  traditional 
lecture-demonstration  form  of  instruction.  The 
computer-based  methods  described  herein. . .of fer 
possibilities  for  lowering  the  cost  of  training. " 
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maintenance  troubleshooting  tasks  versus  more 
traditional  methods  of  instruction.  Subjects  in  both 
experiments  were  advanced  avionics  maintenance  trainees 
enrolled  in  the  final  course  prior  to  FAA 
certification.  The  control  training  media  in  both 
experiments  were  videotaped  programs  supplemented  with 
troubleshooting  reading  assignments  and  on-line 
quizzes.  The  video  programs  included  highly 
context-specific  information  on  the  transfer  tasks. 
There  were  two  experimental  training  conditions  in 
Experiment  1:  one  group,  TASK,  performed  complex 
context-  free  problem-solving  tasks,  while  the  other, 
FAULT,  performed  a  mixture  of  context-free  tasks  and 
tasks  which  provided  some  context-specific  information. 
In  Experiment  2,  there  was  only  one  experimental  group, 
TASK/FAULT,  whose  instruction  hybridized  the 
context-free  TASK  with  the  context-specific  FAULT 
program,  to  emphasize  the  best  features  of  each. 


The  transfer  testing  in  both  experiments  consisted  of 
five  real  troubleshooting  problems  on  live  aircraft 
engines.  Performance  was  measured  in  terms  of  (1) 
"average  performance  index, "  an  average  of  scores  given 
by  the  evaluator  to  each  separate  trainee  action  on  a 
1-5  scale  with  5  being  best;  (2)  "adjusted  time";  (3) 
"evaluator's  rating",  an  overall  test  score  assigned  by 
the  evaluator  at  the  conclusion  of  each  problem. 

b.  Results: 

(1)  Key  Data: 


EXP  1  Average  Performance  Index  for  Each  Problem 


Problem 

TASK 

FAULT 

VIDEO 

Spark  Plug 

3.62 

3.93 

4.29  * 

Secondary  Wire 

3  .76 

3.74 

4.52  * 

Fuel  Obstruction 

4.37 

4.32 

4.42 

Starter  Lead 

4.07 

4.12 

4.29 

Oil  Pressure 

3.99 

3.77 

4.32  * 

*  problems 

explicity 

shown  in 

i  Video  program 

EXP  2  Average 

Performance  Index 

for  Each  Probli 

Problem 

TASK/FAULT 

VIDEO 

Spark  Plug 

4.20 

4.62  * 

Secondary  Wire 

3.82 

4.26 

Fuel  Obstruction 

4.11 

4.10 

Starter  Lead 

3.78 

3.99 

Oil  Pressure 

3.74 

3.86  * 

*  problems 

explicity 

shown  in 

Video  program 
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.  Description  of  Measurements  and  Ratings: 

(1)  Average  performance  index  (reflects  average 
quality  of  sequence  of  actions  taken  by  subject, 
scored  on  standardized  observation  forms) 

(2)  Adjusted  time  (real  time  to  solution  adjusted 
to  manufacturer's  labor  time  schedule) 

(3)  Evaluator's  rating:  overall  performance  score 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 


g.  Statistical  Methods:  not  specified 


h. 


Variables  Being  Manipulated: 

(1)  Training  Devices: 

EXPERIMENT  1 

(a)  Computer  Simulation  (experimental  groups) 

i.  TASK:  complex  variations  of 
context-free  diagnostic  tasks 

ii.  FAULT:  context-specific 
simulations 

(b)  Videotape  and  written  (Control  group) 


EXPERIMENT  2 

(a)  Computer  simulation  combining  both  TASK 
and  FAULT  (from  Experiment  1)  with 
modifications  (Experimental  group) 

(b)  Same  as  Experiment  1  (Control  group) 

(2)  Fidelity  Levels: 

Physical  Functional 

Unknown  Low 

Unknown  Medium 

Medium-low  Medium-low 
of  Task/Skill  Required:  maintenance, 
psychomotor,  perceptual,  procedural, 


TASK 
FAULT 
VIDEO 

(3)  Type 
cognitive, 
part-task 

(4)  Task  Difficulty: 


medium-high 


i.  Stage  of  Training:  skill;  transition;  advanced 


j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 


l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  specified 


Abstract : 


a.  Study  Synopsis:  Two  transfer-of- training 
experiments  were  conducted  to  evaluate  the  training 
effectiveness  of  computer  simulations  of  avoinics 


g.  Statistical  Methods:  ANOVA 

h.  Variables  being  manipulated: 

(1)  Training  Devices: 

(a)  Advanced  Simulator  for  Undergraduate 
Pilot  Training 

(b)  T-37  jet  aircraft  (test  device) 

(2)  Fidelity  Levels: 

(a)  Physical:  (of  simulator)  high 

(b)  Functional:  (of  simulator)  high 

(3)  Type  of  Task/Skill  Required:  aircraft 
piloting;  operations;  cognitive,  psychomotor; 
perceptual;  procedural;  part-task 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  familiarization;  skill 

j.  Trainee  Sophistication:  novice  to  intermediate 

k.  Incorporation  of  device  into  P.O.I.;  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  User  of  Instructional  Features: 

(1)  Intensity:  unspecified,  presumably  intensive 

(2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  the  investigation 
was  to  determine  the  effectiveness  of  ASPT  spin 
recovery  pretraining  in  reducing  the  stress  experienced 
during  the  first  in-flight  exposure  to  power-on  stalls 
and  spin  recoveries  during  T-37  pilot  training. 

Twenty  USAF  student  pilots  were  placed  into  matched 
pairs  according  to  simulator  flying  ability  and  the 
members  of  each  pair  were  randomly  assigned  to  separate 
groups.  Th  experimental  group  received  80  minutes  of 
ASPT  practice  on  power-on  stalls  and  spin  recoveries 
prior  to  the  first  SPIN  ride  in  the  aircraft;  the 
control  group  did  not  recieve  simulation  practice  on 
power-on  stalls  and  spin  recoveries,  but  both  groups 
experienced  ASPT  training  for  take-offs,  60  degree 
turns,  slow  flight,  approaches,  and  landings.  Urine 
samples  of  the  pilots  were  taken  shortly  before  and 
after  the  flight  and  analyzed  for  quantities  of 
catecholamines--  epinephrine  and  norepinephrine — which 
are  indicators  of  emotional  stress.  The  flight 
excretions  were  compared  with  the  same  pilots' 
excretions  during  a  non-stressf ul  period  prior  to  the 
aircarft  flight. 
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b.  Results: 


(1)  Key  Data: 

Comparison  of  Experimental  and  Control  Groups 
Mean  +  S.E.M. 


Condi tion/Vari able 

Experimental 

Control 

P= 

Basal 

Epinephrine  (ng/min) 

4.4 

+ 

0.7 

3.8 

+ 

0.6 

0.523 

Norepinephrine 

37.6 

+ 

4.5 

34  .8 

+ 

3.9 

0.649 

Catecholamine 

42.0 

+ 

4.7 

38.6 

+ 

3.7 

0.589 

NE/E  Ratio 

10.6 

+ 

2.2 

11.9 

+ 

2.4 

0.677 

Spin 

Epinephrine  (ng/min) 

20.1 

+ 

3.2 

38.4 

+ 

8.0 

0.047 

Norepinephrine 

58.7 

+ 

6.7 

43.9 

+ 

6.2 

0.120 

Catecholamine 

78.9 

+ 

8.7 

82.3 

+ 

9.7 

0.795 

NE/E  Ratio 

3.4 

+ 

0.5 

1.5 

+ 

0.3 

0.004 

Performance  Score 

29.8 

+ 

0.9 

29.4 

+ 

1.2 

0.794 

(2)  Verbal  Description:  The  catecholamine 
excretion  means  for  the  experimental  and  control 
f)  •  groups  did  not  differ  significantly.  However,  the 

difference  in  the  ratios  of  norepinephrine  to 
epinephrine  was  significant  at  the  the  0.004  level 
of  confidence. 

c.  Authors'  Conclusions:  The  significant  difference 
in  the  NE/E  ratios  for  the  two  groups  has  important 
implications.  Epinephrine  excretion  is  sensitive  to 
emotional  arousal  and  has  been  reported  to  correlate 
with  feelings  of  anxiety  and  apprehension.  High 
levels...  have  been  shown  to  accompany  mental 
excitement,  confusion,  and  tremor,  all  of  which 
indicate  lack  of  control  and  could  adversely  affect 
piloting  abilities.  Norepinephrine  excretion,  however, 
has  been  shown  to  rise  with  physical  and  mental  stress 
where  events  are  under  the  control  of  the  subject. 

Thus  it  appears  that  ASPT  exposure  and  practice  on 
power-on  stalls  and  spin  recoveries  results  in  a  stress 
response  of  a  somewhat  different  nature  wherein  lower 
emotional  arousal  and  perhaps  a  greater  amount  of 
mental  work  are  experienced  during  the  initial  SPIN 
exposure  in  the  aircraft. 
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1.  Author:  Lintern,  Gavan 


2.  Title:  Transfer  of  Landing  Skill  after  Training  with 
Supplementary  Visual  Cues 

3.  Source:  Human  Factors  ,  1980,  22(1),  81-88 

4.  Topic  Keywords:  Aircraft  Flight  Simulation. ; 

Visual  Cues  ;  Landing  Training  ;  Transfer  of  Training  . 

5.  Short  Summary:  A  simulator-to-simulator ,  transfer  of 
training  experiment  showed  that  adaptive  augmented-  feedback 
simulator  training  had  a  significantly  positive  effect  on 
airplane  landing  performance. 

6.  Device:  Singer-Link  GAT-2  general  function  trainer  with 
closed-loop  visual  display  system  (GAT/VDS) 

7.  Institutions: 

a.  Sponsor:  Life  Sciences  Program, 

Air  Force  Office  of  Scientific  Research 

b.  Performing  Organization:  Aviation  Research 
Laboratory,  Institute  of  Aviation, 

University  of  Illinois  at  Urbana-Champaign 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  4 

b.  Description  of  Groups: 

(1)  Subjects: 

(a)  12  flight-naive  males  18-30  years  old 

(b)  12  same 

(c)  12  same 

(2)  Controls:  12  same 

c.  Tests  or  Trials/Timing:  20  in-process  criterion 
trials,  and  (for  3  subjects)  post-experiment 
checkflight  on  aircraft 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Unspecified  method  of  scoring  criterion  trials 
in  simulator 

(2)  Flight  instructor  ratings  (aircraft  flight, 
post-  training) 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 
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g.  Statistical  methods:  not  specified 


h.  Variables  being  manipulated: 

(1)  Training  Device:  Singer-Link  GAT-2  general 
function  trainer  with  closed-loop  visual  display 
system  (GAT/VDS) 

(a)  Constant  augmentation  cues 

(b)  Adaptive  augmentation  cues 

(c)  Flight  path  tracking  cues 

(d)  Primary  control  (no  supplementary  visual 
cues) 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified,  presumably 
medium-high 

(b)  Functional:  medium-high  to  high 

(3)  Type  of  Task/Skill  Required:  cognitive, 
psychomotor,  motor,  perceptual,  procedural,  part- 
task 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  introduction,  familiarization, 
skill 

j.  Trainee  Sophistication:  novice 

k.  Incorportion  of  device  into  P.O.I.:  not  specified 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified 

(2)  Features  used:  Augmented  feedback;  adaptive 
syllabus 

Abstract : 

a.  Study  Synopsis:  This  study,  using  an  aircraft 
simulator  with  a  variety  of  available  closed-loop, 
computer  generated  visual  displays,  examined  adaptive 
training,  and  secondarily  the  applicability  of  an 
augmented  feedback  technique  to  the  learning  of 
aircraft  flight  tasks,  specifically,  the  landing  task. 

Adaptive  training  was  defined  as  "a  technique  in  which 
the  training  scehdule  is  individualized  by  varying  task 
difficulty  in  a  way  that  is  related  to  the  student's 
speed  of  learning."  The  augmented  feedback  technique 
was  felt  to  most  closely  represent  the 
adaptive- training  technique  out  of  the 
computer-generated  displays  available  on  the  device. 

After  initial  training  on  the  device,  4  groups  of  12 


flight-naive  subjects  each  performed  trial 
approach-and-  landings  on  the  simulator,  each  using  one 
of  the  4  displays  available:  (1)  Primary  Control 
Group,  a  computer-generated  skeletal  airport  scene 
consisting  of  a  horizon  line  and  a  runway  outline  with 
centerline  and  aiming  bar;  (2)  Flightpath  Tracking 
Group,  a  fixed  cross  and  a  moving  square  projected  onto 
the  screen;  (3)  Constant  Augmented-Feedback  Group,  in 
addition  to  airport  graphics  used  for  the  Primary 
Control  Group,  command  guidance  cues  that  showed  the 
desired  flightpath  during  the  approach,  proper  height 
for  flare,  and  an  extended  centerline;  (4)  Adaptive 
Augmented-Feedback  Group,  same  as  the  latter  except 
that  the  command  guidance  cues  appeared  only  when  the 
subject  deviated  from  specified  performance  criteria. 

Some  subjects  also  subsequently  performed  real- 
aircraft  landings  and  were  scored  by  Instructor  Pilots. 

b.  Results: 


(1)  Key  Data 


Group  Statistics  and  Comparisons  among  training 
treatments  for  frequencies  of  criterion  performances 
during  Landing  Approach  Trials  21  to  36 


Primary  Constant  Adaptive  Flight  path 
Control  Augmentation  Augmentation  Tracking 
(PC)  (CA)  (AA)  (FT) 


Mean  4.58  4.75 

S.D.  2.50  2.93 


8.33  3.58 

2.50  2.99 


CA  vs  PC 
AA  vs  PC 
AA  vs  CA 
FT  VS  PC 


Comparisons 
Mean  Diff.  df 


0.17  44 

3.75  44 

3.58  44 

-1.00  44 


t  P 

0.15  0.882 

3.35  0.002 

3.20  0.003 

-0.89  0.376 


(2)  Verbal  Description:  During  the  first  20 
trials,  the  adaptive  augmented-f eedback  group 
produced  the  best  performance,  although  neither  of 
the  comparisons  with  the  constant 
augmented-f eedback  group  or  the  primary  control 
group  were  statistically  reliable.  During  trials 
21  to  36  the  adaptive  augmented-  feedback  group 
outperformed  both  the  constant  augmented-feedback 
group  and  the  primary  control  group  (p‘0.01), 
shown  above. 


Unassisted  landings  and  subjective  performance 
ratings  by  qualified  flight  instructors  showed 
that  the  students  who  attempted  to  land  the 
airplane  after  their  simulator  landing  instruction 
performed  better  on  unassisted  trials  and  on 
subjective  ratings. 

c.  Author's  Conclusions:  Supplementary  visual  cues 
appeared  to  facilitate  approach  and  landing  performance 
in  the  simulator,  at  least  when  presented  in  the 
adaptive  mode.  Some  adaptive  augmented-f eedback 
manipulations  can  be  more  effective  than  fixed-task 
methods  for  teaching  perceptual-motor  tasks  with  poor 
intrinsic  feedback  such  as  landing  an  airplane. 

Within  the  limitations  of  the  test  that  was  undertaken, 
the  simulator  training  had  a  strong  and  statistically 
reliable  influence  on  the  airplane  landing  work.  It 
helped  students  on  both  frequency  of  unassisted 
landings  and  subjective  ratings  of  performance  by 
qualified  flight  instructors. 
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1.  Author:  Martin,  Elizabeth  L. 

2.  Title:  Training  Effectiveness  of  Platform  Motion: 

Review  of  Motion  Research  Involving  the  Advanced  Simulator 
for  Pilot  Training  and  the  Simulator  for  Air-to-Air  Combat 

3.  Source:  AFHRL-TR-7 9-51 ,  February  1981 

4.  Topic  Keywords:  Platform  Motion  ;  Flight  Simulation  ; 
Motion  Simulation  ;  Transfer  of  Training  ; 

Training  Effectiveness  . 

5.  Short  Summary:  A  review  of  six  transfer-of-  training 
studies  involving  the  use  of  platform  motion  in  the  training 
simulator  finds  that  the  addition  of  task  correlated 
platform  motion  cueing  results  in  negligible  transfer-of- 
training  for  initial  jet  piloting  skills. 

6.  Devices: 

a.  Advanced  Simulator  for  Pilot  Training  (ASPT) 

b.  Simulator  for  Air-to-Air  Combat  (SAAC) 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Operations  Training 
Division,  Air  Force  Human  Resources  Laboratory, 

Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Six  experiments  are  cited;  Roman 
Numerals  below  follow  text: 

I.  STUDY  I 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  8  Pre-flight  Air  Force  UPT 
Students  (simulator  motion) 

(2)  Subjects:  8  same  (without  simulator 
motion) 

(3)  Controls:  8  same  (without  simulator 
training) 

c.  Tests  or  Trials/Timing:  In-process  checkrides 

d.  Number  of  Different  Types  of  Measures:  2 
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e.  Description  of  Measurements  and  Ratings: 

(1)  IP  grades 

(2)  Automatic  pilot  performance  data 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(.1)  Training  Devices:  Advanced  Simulator  for 
Pilot  Training 

(2)  Fidelity  Levels:  Physical  Functional 

With  Motion  High  Very  High 

No  Motion  High  High 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual, 
procedural,  part-task,  whole-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  introduction;  skill 

j.  Trainee  Sophistication:  intermediate/novice 

k.  Incorporation  of  Device  into  P.O.I.: 
lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  discussed 
STUDY  II 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  12  post-solo  UPT  students 
(with  simulator  motion 

(2)  Subjects:  12  same  (without  motion) 

(3)  Controls:  12  same  (without  simulator 
training) 

c.  Tests  or  Trials/Timing:  In-process  checkrides 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  IP  overall  grade 

(2)  IP  grade  per  maneuver 

f. -m.  Same  as  for  STUDY  I,  except  for  the 
following  changes: 
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i.  Stage  of  Training:  Skill 

j.  Trainee  Sophistication:  Intermediate/high 
III.  STUDY  III 

a.  Number  of  Groups:  4 

b.  Description  of  Groups: 

(1)  Subjects:  8  Pre-flight  UPT  students 

(motion/full  f ield-of- view) 

(2)  Subjects:  8  same  (motion/limited 

f  ield-of-view) 

(3)  Subjects:  8  same  (no  motion/full  F.O.V.) 

(4)  Subjects:  8  same  (no  motion/limited 

F.O.V. ) 

No  Controls 

c.  Tests  or  Trials/Timing:  post-checkride 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  IP 

W)  •  Grade  by  task 

f. -m.  Same  as  for  STUDY  I,  except  for  the 
following  changes: 

h.  Variables  being  manipulated: 

(1)  Training  Devices:  Advanced  Simulator  for 
Pilot  Training 

(2)  Fidelity  Levels:  Physical  Functional 

Motion/Full  FOV  High  Very  High 

Motion/Limited  FOV  Med. -high  Very  High 

No  Motion/Full  FOV  High  High 

No  Motion/Limited  FOV  Med. -high  High 

IV.  STUDY  IV 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  8  T-37  phase  UPT  students 
(with  motion) 

(2)  Subjects:  8  same  (without  motion) 

c.  Tests  or  Trials/Timing:  In-process  trials 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  Time 
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to  complete  training  by  hours  in  simulator  and 
aircraft 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Device:  Advanced  Simulator  for 
Pilot  Training 

(2)  Fidelity  Levels:  Physical  Functional 

With  Motion  High  Very  High 

No  Motion  High  High 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual, 
procedural,  part-task,  whole-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Skill 

j.  Trainee  Sophistication:  Intermediate/High 

k. -m.  Same  as  for  STUDY  I 
V.  STUDY  V 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  8  Replacement  Training  Unit 

Students 

(2)  Subjects:  8  same 

(3)  Controls:  6  same 

c.  Tests  or  Trials/Timing:  In-process  checkrides 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  IP  Overall  grades 

(2)  IP  grade  by  maneuver 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Simulator  for 
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Air-to-Air  Combat 

(2)  Fidelity  Levels:  Physical  Functional 


With  Motion  High  High 

No  Motion  High  Medium-High 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychoniotor ,  perceptual, 
procedural,  part-task,  whole-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Skill;  transition 

j.  Trainee  Sophistication:  High 

k. -m.  Same  as  for  STUDY  I 


VI.  STUDY  VI 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  8  Graduate  student  pilots  with 
fighter  lead-in  training  (with  motion) 

(2)  Subjects:  8  same  (without  motion) 

(3)  Controls:  8  same  (without  simulator 
training) 

c.  Tests  or  Trials/Timing:  Post-tests 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Circular  bombing  error 

(2)  Number  of  qualifying  bombs 

(3)  IP  Grade 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 


g.  Statistical  Methods:  not  specified 


h.  Variables  Being  Manipulated: 

(1)  Training  Device:  Advanced  Simulator  for 
Pilot  Training 

(2)  Fidelity  Levels:  Physical  Functional 


With  Motion 
No  Motion 


High  Very  High 

High  High 


(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual, 
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procedural,  part-task 
(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Skill 

j.  Trainee  Sophistication:  Expert 

k.  Incorporation  of  Device  into  P.O.I.: 

Lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  discussed 
9.  Abstract: 

a.  Study  Synopsis:  This  study  reviewed  five 
experiments  performed  with  the  Air  Force  Advanced 
Simulator  for  Pilot  Training  (ASPT)  and  one  experiment 
performed  with  the  Simulator  for  Air-to-Air  Combat 
(SAAC) ,  which  measured  the  effects  of  simulated  6 
degrees  of  freedom  motion  during  training  of  jet  pilots 
on  transfer  performance  in  the  aircraft.  Five  of  these 
experiments  also  had  control  groups  who  received  no 
training  in  the  advanced  simulators,  in  order  to 
determine  the  effectiveness  of  training  in  simulators 
regardless  of  motion.  The  studies  are  summarized  as 
follows: 

Study  I  (Basic  Contact) :  Three  groups  of  8  pre-  flight 
UPT  students  each  were  differently  trained  for  basic 
contact  maneuvers  in  T-37  jet  trainer  aircraft. 

Control  group  received  standard  syllabus  with  simulated 
pretraining  in  T-4  non-visual  instrument  trainer. 
Experimental  groups  received  experimental  syllabus 
training  with  the  Advanced  Simulator  for  Pilot 
Training,  identical  except  that  one  group  had  platform 
motion  in  the  ASPT,  the  other  did  not.  Proficiency 
trials  were  two  special  T-37  checkflights  during  the 
course  of  training,  plus  task  frequency  data  on  all 
missions  through  solo  sorties. 

Results:  No  consistent  differences  were  found  between 

the  transfer  performances  of  the  two  (motion  and  no¬ 
motion)  ASPT  groups.  The  ASPT  groups  performed  better 
in  transfer  than  the  control  group. 

Study  II  (Aerobatics):  Three  groups  of  12  post-solo 
UPT  students  each  were  differently  trained  for  flying 
aerobatics  exercises  in  the  T-37  jet  trainer  aircraft. 
Control  group  received  standard  syllabus  training. 
Experimental  groups  received  ASPT  training  in  addition 
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to  aircraft  instruction;  one  group  trained  in  the  ASPT 
with  motion,  the  other  without.  Performance  was 
evaluated  by  instructor  pilots  both  in  aircraft  and  in 
the  simulator,  on  specific  parameter  values  per 
maneuver,  and  an  overall  rating. 

Results:  On  transfer,  the  ASPT  groups  outperformed  the 
control  group  in  one  maneuver.  There  were  no 
differences  between  the  ASPT  groups  trained  with  motion 
and  without  motion. 

Study  III  (Motion/Visual  Interaction):  Four  groups  of 
8  pre-flight  UPT  students  each  were  differently  trained 
in  the  ASPT  for  flying  four  basic  tasks.  Training 
treatments  were  as  follows:  (1)  motion/full  field  of 
view;  (2)  motion/limited  field  of  view;  (3)  no 
motion/full  field  of  view;  (4)  no  motion/limited  field 
of  view. 

Results:  No  statistically  reliable  differences  were 

observed  between  the  groups  in  transfer  performance  in 
the  aircraft. 

Study  IV  (Motion  vs.  No-Motion  in  UPT  Syllabus  Study) : 
Two  groups  of  T-37  phase  UPT  students  were  differently 
trained  in  the  ASPT  during  an  experimental  syllabus; 
one  group  with  motion,  the  other  without.  Students 
were  advanced  on  the  basis  of  an  IP's  judgement  of 
proficiency  for  the  aircraft.  The  amount  of  time  spent 
in  simulator  training  to  reach  the  IP-judged 
proficiency  level  was  the  measure  of  effectiveness. 

Results:  No  significant  differences  in  training  time 

to  reach  proficiency  appeared  between  the  two  groups. 

Study  V  (Motion  vs.  No-Motion--Basie  Fighter 
Maneuvering) :  Three  groups  of  Replacement  Training 

Unit  students  transitioning  to  the  F-4  were  differently 
trained;  the  Control  Group  of  6  received  training  in 
the  SAAC,  one  with  motion,  one  without.  Students' 
performance  in  the  F-4  aircraft  was  graded  by 
Instructor  Pilots. 

Results:  There  were  no  significant  differences  between 

any  of  the  groups  in  overall  mission  performance 
ratings  in  the  aircraft,  although  the  control  group 
outperformed  the  SAAC-trained  groups  in  some  maneuvers. 

Study  VI  (Motion  vs.  No-Motion — Air-to-Surf ace  Weapons 
Delivery) :  Three  groups  of  8  graduate  students  with 

fighter  lead-in  experience  each,  were  trained 
differently  to  perform  dive  bombing  tasks  in  an  F-5B 


aircraft.  The  control  group  received  no  ASPT  training; 
the  two  experimental  groups  received  training  on  an 
ASPT  configured  as  a  T-37,  one  with  motion,  one 
without. 

Results:  The  experimental  ASPT-trained  groups 

outperformed  the  control  group  in  dive  bombing  runs  in 
the  aircraft;  however,  there  were  no  differences 
between  the  performances  of  the  Motion  and  No-Motion 
ASPT  groups. 

b.  Results:  Given  on  a  per-study  basis  above. 


c.  Author's  Conclusions:  Simulator  training  generally 
led  to  better  transfer  performance  with  the  exception 
of  the  Basic  Fighter  maneuvers  taught  in  the  SAAC; 
this  exception  indicated  a  failure  of  the  specific 
training  methodology.  In  the  case  of  the  bombing  task 
experiment  (Study  VI),  the  superiority  of  the 
experimental  groups,  trained  in  a  device  which 
simulated  a  different  aircraft  from  the  transfer 
aircraft,  indicated  that  effective  transfer  of  training 
on  this  task  did  not  require  a  maximum-fidelity 
simulator . 


The  author  makes  the  following  general  conclusions: 

(1)  Platform  motion  has  little  or  no  demonstrated 
positive  effect  on  transfer  of  training,  at  least  for 
novice  jet  pilots  acquiring  basic  contact  skills. 

(2)  Platform  motion  has  a  small  effect  on  the 
performance  of  experienced  pilots  in  the  simulator. 

(3)  Platform  motion  has  the  most  potential  for 
enhancing  simulator  training  on  specific  tasks 
requiring  control  in  a  marginally  stable  condition. 

Existing  data  do  not  support  procurement  of 
sophisticated  six-post  synergistic  platform-motion 
systems  for  pilot  contact  skill  acquisition;  and 
existing  simulators  for  pilot  training  possessing 
synergistic  platform-motion  systems  can  be  equally 
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multiple  regression 


g.  Statistical  Methods:  AMC'i. * * * * * 7 * 9A; 
analysis 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Modularized  Synthetic  Sonar 
Trainer 

Experiment  1:  various  combinations  of  complexity, 
feedback  and  embeddedness 
Experiment  2  : 

(a)  Hot  panel  -  dynamic  mode 

(b)  Cold  panel  -  inoperative  mode 

(c)  Pictorial  representation  -  photograph  of 
panel 


(2)  Fidelity  Levels:  Physical  Functional 

Experiment  1  varied  varied 

Medium-High  Medium-high 


Experiment  2 

HOT 

COLD 

PICTORIAL 


High  High 

High  Low 

Medium-Low  Low 


(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  varied  Medium-Low  to 
Medium-High  on  acquisition;  Medium  on  transfer 


i.  Stage  of  Training:  introduction, 

f amil iar ization--not  strictly  applicable  since  this  was 
experimental  only 


j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

9 .  Abstract : 


a.  Study  Synopsis:  This  study  concluded  a  three-phase 
series  which  sought  to  establish  quantitative  task 
indices  for  use  in  forecasting  training  device 
effectiveness.  An  earlier  phase  validated  the 
relevance  of  certain  task  indices  to  acquisition;  the 
present  study  was  aimed  at  (1)  applying  a  battery  of 
these  indices  to  transfer  of  training,  and  (2) 


268 


1.  Authors:  Mirabella,  Angelo,  &  Wheaton,  George  R. 

2.  Title:  Effects  of  Task  Index  Variations  on  Transfer  of 
Training  Criteria 

3.  Source:  NAVTRAEQUIPCEN  Technical  Report  72-C-0126-1, 
January  1974 

4.  Topic  Keywords:  Quantitative  Task  Indices  ; 

Transfer  of  Training  ;  Sonar  Console  Simulation  . 

5.  Short  Summary:  The  concluding  study  in  a  series  aimed 
at  establishing  quantitative  task  indices  to  forecast 
training  device  effectiveness  finds  some  practical  value  for 
the  indices,  and  presents  evidence  for  interaction  between 
task  indices  and  training  methods  involving  different 
degrees  of  simulation  fidelity. 

6.  Devices:  Modularized  synthetic  sonar  trainer  with 
capability  for  changing  modules  between  "simple"  and 
"complex",  for  manipulating  feedback  and  "embeddedness"  of 
tasks. 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization:  Silver  Spring,  MD, 
American  Institutes  for  Research  , 

8.  Type  of  Article:  Experiment  (Two) 

a.  Number  of  Groups:  Experiment  1:  20;  Experiment 

2:18 

b.  Description  of  Groups:  For  both  experiments,  each 
group  consisted  of  5  university-connected  personnel 
from  Washington,  DC  area;  randomly  assigned 

c.  Tests  or  Trials/Timing:  in-process  trials; 
post-test  for  transfer 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Time  to  complete  task 

(2)  Errors  on  task 

f.  Experimental  Setting/Training  Context:  laboratory, 
hands-on 
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the  simulation  group. 


The  students'  evaluations  of  their  improvement  showed 
no  difference  between  groups. 

c.  Authors'  Conclusions:  Simulators  seem  an 
inefficient  way  simply  to  communicate  safety 
information.  Any  difference  between  the  two  groups  on 
the  knowledge  test  can  be  attributed  to  differential 
emphasis  upon  simulator  and  lecture  subjects...  that 
occurred  despite  our  efforts  to  equalize  them. 

The  same  may  well  hold  true  for  attitudes. 

Potential  for  effective  simulated  driver  training  lies 
in  the  capability  of  the  simulator  to  complete  a 
stimul us- response  connection.  What  is  needed  to 
exploit  the  full  capabilities  of  simulation  is  (1)  far 
greater  practice  on  responses  to  be  acquired;  (2) 
periodic  retesting  to  sustain  response  tendencies  at  a 
high  level;  and  (3)  research  to  determine  appropriate 
emergency  responses  as  well  as  the  limits  of  effective 
stimulus  and  response  generalization. 
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(2)  Fidelity  Levels:  not  specified;  presumably 
low 

(3)  Type  of  Task/Skill  Required:  automobile 
driving;  cognitive;  psychomotor 

(4)  Task  Difficulty:  Low 

i.  Stage  of  Training:  Refresher 

j.  Trainee  Sophistication:  Intermediate/Expert  (all 
were  experienced  drivers) 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  specified 

m.  Use  of  Instructional  Features:  not  specified 
9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  the  study  was  to 
evaluate  the  comparative  effectiveness  of  a  simulator 
in  teaching  safe  driving  knowledge  and  habits,  versus 
conventional  driver  training  media.  Twelve  classes, 
totaling  238  enlisted  men,  were  administered  a  20-hour 
driver  improvement  program.  Half  of  the  classes 
received  eight  hours  of  simulator  instruction  while  the 
remaining  half  received  the  same  content  taught  by 
conventional  media  including  films,  and  group 
discussions. 

Driving  knowledges  were  measured  after  the  training  by 
a  50-item  multiple  choice  test  covering  both  simulator 
and  lecture  material.  Also,  a  test  drive  on  the 
simulator  was  given  to  all  subjects  and  controls; 
controls  were  given  familiarization  with  the  device 
prior  to  testing.  (Note:  the  reviewer  infers  that  a 
test  drive  was  the  second  test  medium;  the  authors 
never  said  so  explicitly). 

Students  also  made  a  self-evaluation  of  their  own 
improvement  during  the  course. 

b.  Results:  The  knowledge  test  revealed  no  truly 
significant  differences.  Whatever  differences  exist 
(at  the  .10  to  .15  level),  favor  the  simulator  group 
for  covered  in  the  simulator  program,  the  conventional 
group  for  lecture  material. 

The  results  for  driving  habits  paralleled  the  empnasis 
given  to  them  in  the  simulator  programs.  Those  tnat 
were  practiced  most  often...  showed  differences  of 
both  statistical  and  practical  significance  in  favor  of 
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1.  Authors:  McKnight,  A.  James,  &  Hunter,  Harold  G. 


2.  Title:  An  Experimental  Evaluation  of  a  Driver  Simulator 
for  Safety  Training 

3.  Source:  HumRRO  Professional  Paper  9-66,  December  1966 

4.  Topic  Keywords:  Stimulus-Response  Connection  ; 
Generalization  Gradients  ;  Simulation  ; 

Training  Effectiveness  . 

5.  Short  Summary:  Initial  studies  of  the  use  of  simulators 
in  automobile  driver  training  conclude  that  then-  existing 
moving-picture  driving  simulators  do  not  merit  full  scale 
long-term  study  until  more  research  is  done  on  the  limits  of 
effective  stimulus- response  generalization. 

6.  Device:  Motion-picture  automobile  driving  simulator 

7.  Institutions: 

a.  Sponsor:  U.S.  Continental  Army  Command 

b.  Performing  Organization:  HumRRO  Division  No.  1, 
Alexandria,  VA 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  12 

b.  Description  of  Groups: 

{ 1 ) — ( 6 )  Subjects:  Approximately  14  each  group, 
enlisted  military  personnel  holding  driver's 
licenses 

(7) — (12)  Controls:  Same 

c.  Tests  or  Trials/Timing:  2  post-tests  (up  to  2  1/2 
mo.s  after  training) 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Multiple-choice  knowledge  questions 

(2)  Test  drive  in  simulator  (reviewer's  inference) 

f.  Experimental  Setting/Training  Context:  experiment 
at  military  base;  driver's  education  program 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Device:  as  above,  section  6 
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experimental  subjects  and  controls,  except  that  in  the 
device- training  portion  of  the  radar  maintenance  block, 
the  subjects  were  trained  on  the  ECU  only,  the 
controls  on  the  AET  only.  Subjects  numbered  15  and 
controls,  24,  with  the  controls  having  more  prior 
experience  with  radar  maintenance  (how  much  more  was 
not  specified) .  Two  immediate  post-tests  were  given  to 
subjects  and  controls  after  the  radar  maintenance 
block:  a  normal-system-  operation  checklist  and  a  set 

of  eight  simple  malfunction  troubleshooting  problems. 

b.  Results: 

(1)  Key  Data:  2  of  39  students  failed  to  reach 
criterion  performance  (both  groups  combined) . 

Mean  score:  90.5  out  of  95  checklist  items  (both 
groups  combined) 

(2)  Verbal  Description:  No  significant 
differences  were  found  between  simulator- trained 
subjects  and  actual-  equipment- trained  controls  on 
the  post-tests. 

Questionnaires  given  to  instructors  and  students 
found  the  simulator  generally  acceptable  to  both; 
although  those  instructors  who  had  not 
participated  in  the  development  of  the  simulator 
were  cautious  in  recommending  it  (they  had  less 
knowledge  of  its  capabilities).  "In  the  course  of 
evaluating  the  students  on  the  normal  operations 
checkout,  the  instructors  felt  that  those  students 
trained  on  the  simulator  exhibited  a  more  logical 
or  rational  approach  to  the  checkout  procedure." 

c.  Authors'  Conclusions:  The  ECU  was  found  at  least 
equally  as  effective  as  the  AET  in  training  the 
selected  radar  maintenance  skills  in  a  refresher 
course.  The  students  trained  on  the  simulator  (those 
with  little  or  no  previous  related  experience) 
performed  as  well  as  those  training  on  the  trainer 
(those  with  previous  radar  system  experience). 

Evidence  from  instructor  comments  suggests  that  perhaps 
the  unique  training  capabilities  of  the  simulator 
enable  the  student  to  adopt  a  more  appropriate 
"cognitive  style"  for  this  kind  of  task. 


f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 

g.  Statistical  Methods:  Not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  ECU  computerized  multiply-programmable 
simulator 

(b)  Actual  Equipment  Trainer,  modified  for 
training 

(2)  Fidelity  Levels: 

(a)  Physical:  ECII,  Low-Medium;  AET,  High 

(b)  Functional:  ECII,  High;  AET,  High 

(3)  Type  of  Task/Skill  Required:  maintenance, 
cognitive,  procedural 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  Refresher 

j.  Trainee  Sophistication:  Intermediate  to  high 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  fair,  as  assessed  by 
questionnaire 

(2)  Students:  noncommittal 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  unspecified;  probably  incomplete 

(2)  Features  used:  freeze  capability, 
restart/resequence  capability;  malfunction 
selection;  number/quality  of  responses;  probably 
cue  enchancement  features;  others  not  specified 

Abstract : 

a.  Study  Synopsis:  The  study  was  designed  to  compare 
the  effectiveness  of  the  ECII  computerized  multiply- 
programmable  equipment  with  that  of  the  Actual 
Equipment  Trainer  then  in  standard  use  in  a  refresher 
maintenance  training  course  for  Air  Force  and  Air 
National  Guard  personnel.  The  ECII,  while  of 
considerably  lower  physical  fidelity  than  the  AET,  had 
a  number  of  instructional  features  not  available  on  the 
AET.  A  cost  analysis  of  the  two  devices  was  also  part 
of  this  study,  which  found  that  developing  and 
procuring  the  simulator  was  much  less  expensive  than 
procuring  an  AET. 

The  standard  training  syllabus  was  used  for  both 


1.  Authors:  McGuirk,  Frank  D. ,  &  Pieper,  William  J. 

2.  Title:  Operational  Tryout  of  a  General  Purpose 
Simulator 

3.  Source:  AFHRL-TR-7 3-13 ,  May  1975 

4.  Topic  Keywords:  Troubleshooting  Maintenance  Training  ; 
General  Purpose  Simulator  . 

5.  Short  Summary:  Comparison  of  a  General  Purpose 
Maintenance  Training  Simulator,  ECU  (a  computerized 
multiply-  programmable  training  device)  with  an  Actual 
Equipment  Trainer  modified  slightly  for  training,  showed  the 
General  Purpose  Simulator  at  least  equally  effective  in 
training  students  in  radar  maintenance  tasks,  at  a  fraction 
of  the  cost. 

6.  Devices:  ECU  computerized  multiply-programmable 
maintenance  Training  Simulator;  and  Actual  Equipment 
Trainer,  field  equipment  slightly  modified  for  training 
purposes 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  AFB ,  TX  78235 

b.  Performing  Organization: 

Applied  Science  Associates,  Inc.,  Box  158,  Valencia,  PA 
16059 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  15  airmen  undergoing  retraining  (to 
be  trained  on  simulator) 

(2)  Controls:  24  same  but  with  more  prior  radar 
maintenance  experience  (to  be  trained  on  Actual 
Equipment  Trainer) 

c.  Tests  or  Trials/Timing:  Two  post-tests 
a.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Procedures  Checklist 

(2)  Malfunction  Problem-solving 

(a)  No,  of  trials  to  solution 

(b)  Time  to  solution 
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measures  were  criterion-referenced,  not  merely  control 
input  derivations,  and  so  may  not  be  the  most  sensitive 
to  motion-cue  manipulations;  (4)  with  the  exception  of 
stalls,  motion  cues  were  not  training-relevant  cues  for 
the  tasks  covered  in  this  study;  and  (5)  the  subjects 
were  student  pilots  with  no  previous  jet  flying 
experience,  while  theoretically,  motion  cues  acquired 
meaning  as  a  function  of  experience.  For  whatever 
reason,  motion  cueing  was  not  a  potent  training 
variable.  The  data  failed  to  demonstrate  any 
enchancement  of  training  effectiveness  as  a  result  of 
the  addition  of  platform  motion. 
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(2)  Verbal  Desctription : 

(a) 

On  10 

ASPT 

mission 

profiles  (experimental 

sub j  ects 

only)  , 

there  ’ 

were  not 

significant 

differences  between  the  motion  and  no-motion 
conditions  for  any  of  the  maneuvers  using 


either  the  automated  performance  measures  or 
the  Instructor  Pilot  ratings. 

(b)  On  T-37  aircraft  training  transfer 
evaluations  by  checkflight,  there  were  not 
significant  differences  between  the  motion 
and  no-motion  simulator  training  groups  as 
derived  from  inprocess  data  rides  on 
aircraft;  whereas,  the  performance  of  the 
ASPT-trained  groups  combined  was  superior  to 
the  control  groups  across  all  maneuvers 
(utilizing  a  prior  "t"  tests),  as  determined 
from  task  frequency  data  on  checkflight. 

c.  Authors'  Conclusions:  The  motion  variable  did  not 
have  significant  beneficial  training  value  in  either 
the  simulator  training  phase  of  the  aircraft  training 
phase . 

There  are  several  experimental  design  factors  which  may 
have  contributed  to  these  findings:  (1)  small  sample 
size--8  per  group;  (2)  the  between-  and  within-subj ect 
variability  was  high,  and  there  were  uncontrolled 
factors  which  acted  to  increase  variance,  e.g.,  weather 
conditions,  etc.,  on  sorties;  (3)  the  dependent 
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which  quantitative  data  were  obtained  as  well  as 
Instructor  Pilot  ratings,  and  received  two  special  data 
flights  on  the  aircraft;  both  subjects  and  controls 
received  post-experiment  performance  checkflights 
("pre-solo  syllabus  sorties"),  where  task  frequency 
data  were  recorded  by  Instructor  Pilots. 

Results  established  the  training  value  of  the  Advanced 
Simulator  for  Pilot  Training,  for  the  simulator- trained 
pilots  outperformed  the  aircraft-trained  pilots  in  the 
aircraft  checkflight,  although  it  is  important  to 
remember  that  the  simulator- trained  students  had 
already  received  two  special  data  flights  on  the 
aircraft  prior  to  taking  the  checkflight.  The  data 
failed  to  reveal  any  significant  difference  between  the 
two  simulator-trained  groups  in  performance  on  either 
simulator  or  aircraft,  indicating  that  at  least  for 
this  level  of  experience,  the  addition  of  platform 
motion  in  simulated  training  did  not  enhance  learning. 

b.  Results: 

(1)  Key  Data: 

Descriptive  Statistics  for  ASPT  Performance  Evaluations 

APM  Scores 
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and  controls,  on  checkf lights ;  for  subjects,  on 
ASPT  mission  profiles) 

f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 

g.  Statistical  Methods:  ANOVA;  t-test 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Advanced  Simulator  for 
Pilot  Training  (ASPT) 

(a)  With  6  DOF  platform  motion 

(b)  No  platform  motion 

(2)  Fidelity  Levels: 

(a)  Physical:  High 

(b)  Functional:  High;  higher  for  device 
with  motion 

(3)  Type  of  Task  Required:  Jet  aircraft  flight; 
operations 

(4)  Task  Difficulty:  High 

(5)  Skills  required  by  task:  motor;  perceptual; 
cognitive 

i.  Stage  of  Training:  Introduction;  familiarization; 
skill 

j.  Trainee  Sophistication:  Novice  and  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  unspecified 

(2)  Students:  unspecified 

m.  Use  of  Instructional  Features:  Intensive 
9.  Abstract: 

a.  Study  Synopsis:  A  transf er-of- training  experiment 
was  designed  to  evaluate  the  contributions  of  simulated 
motion  to  training  on  a  high  fidelity  aircraft  flight 
simulator.  A  synergistic  6-degr ee-of-f reedom  platform 
motion  system  was  used  in  the  training  of  basic  jet 
aircraft  contact,  approach,  and  landing  skills.  24 
pilot  trainees  with  no  jet  flight  experience  and  a  mean 
of  28.8  hours  pilot  experience  overall  were  divided 
into  three  training  groups  of  8  members  each:  (1) 
received  t.w-ning  on  simulator  with  motion;  (2) 
received  training  on  simulator  without  motion;  (3) 
received  training  only  on  aircraft  (control  group). 

The  simulator-trained  groups  received  10  instructional 
sorties  in  the  simulator  on  basic  contact  tasks,  for 
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1.  Authors:  Martin,  Elizabeth  L. ,  &  Waag,  Wayne  L. 

2.  Title:  Contributions  of  Platform  Motion  to  Simulator 
Training  Effectiveness;  Study  1  -  Basic  Contact 

3.  Source:  Technical  Report:  AFHRL-TR-7 8-1 5 ,  June  1978 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Flight  Simulation  ;  Platform  Motion  ; 

Training  Effectiveness  . 

5.  Short  Summary:  A  transfer  of  training  study  shows  no 
significant  differences  between  test  performances  (including 
real  aircraft  flight)  of  jet  aircraft  flight  trainees 
trained  on  simulators  with  kinesthetic  and  vestibular  clues 
and  those  trained  on  the  same  device  without  kinesthetic  and 
vestibular  clues.  However,  on  advanced  flight  tasks,  both 
groups  out-performed  a  control  group  that  received  all  its 
training  on  the  aircraft. 

6.  Devices:  Advanced  Simulator  for  Pilot  Training  (ASPT) 

7.  Institutions: 

a.  Sponsor:  Hq  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  Air  Force  Base,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
Air  Force  Human  Resources  Laboratory,  Williams  Air 
Force  Base,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  (ASPT  trained,  with  motion):  8 

undergraduate  pilot  training  students,  minimal 
flying  experience 

(2)  Controls:  8  same 

(3)  Subjects:  (ASPT  trained,  no  motion):  8  same 

c.  Tests  or  Trials/Timing:  10  process  tests  on 
simulator  (experimental  subjects  only);  2  process 
tests  on  aircraft;  1  post-test  on  aircraft 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Automated  Performance  Measures  (with-subj ect 
only,  on  simulator;  for  10-in  process  trials) 

(2)  Instructor  Pilot  ratings  (for  both  subjects 
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base:  Martin  &  Waag,  June  1978;  Nataupski,  et  al . 
November  1979;  and  Woodruff,  et  al . ,  December  1976 
respectively. 


examining  the  effects  of  interaction  between  task 
characteristic  measured  by  the  indices,  and  training 
methods,  specifically  simulation  fidelity. 

Two  experiments  were  conducted  using  a  modularized 
synthetic  sonar  trainer.  Modularization  of  the 
simulator  enabled  the  experiments  to  vary  task 
complexity,  among  of  feedback,  and  the  "embeddedness" 
of  the  task  within  the  console  as  a  whole.  In  both 
experiments,  all  training  and  testing  were  conducted  on 
the  simulator.  In  the  first  experiment,  subjects  were 
trained  in  groups  of  5,  on  one  of  the  20  tasks  which 
differed  over  a  wide  range  of  task  index  values,  and  in 
levels  of  feedback  and  embeddedness;  subjects  were  all 
tested  for  transfer  on  one  task  of  medium  complexity. 

In  the  second  experiment,  three  levels  of  fidelity  were 
employed  in  the  simulator  during  training:  (1)  "hot" 
panel,  high  physical  and  high  functional  fidelity;  (2) 
"cold"  panel  (did  not  operate),  high  physical  but  low 
functional  fidelity;  and  (3)  "pictorial 
representation,"  a  photograph  of  the  console, 
medium-low  physical  fidelity  and  low  functional 
fidelity. 

The  performances  on  training  and  transfer  trials  were 
measured  in  terms  of  time  to  complete  tasks,  and  number 
of  errors. 
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b.  Results:  EXPERIMENT  1 


(1)  Key  Data: 

Multiple  Regression  Analyses  Using  Difference  Scores  to 
Predict  Raw  Time  and  Error  Scores  for  First,  Middle, 


Last  Block  of 

Transfer 

Trials 

% 

Criterion 

R 

R2 

df 

F 

Time 

Scores 

Transfer 
Trials  1-2 
(First  Block) 
Transfer 
Trials  5-6 

.751 

.  56  4 

3,  11 

4  .75 

I 

(Middle  Block 
Transfer 
Trials  9-10 

)  .771 

.  595 

3,  11 

5  .39 

1 

(Last  Block) 

.80  5 

.648 

3,  11 

6  .76 

★ 

Error 

Scores 

Transfer 
Trials  1-2 

.  890 

.793 

3,  11 

14.03 

*★ 

Transfer 
Trials  5-6 

.  914 

.836 

3,  11 

18.67 

*★ 

Transfer 
Trials  9-10 

.82  4 

.67  9 

3,  11 

7  .75 

★ 

Indices  represent  absolute  differences  between 
acquisition  and  transfer  tasks. 

•  d.  <  .025. 

*  p.  <  .01. 

**  p.  <  .001. 

R  =  Multiple  correlation  coefficient 
R2  =  Percentage  of  variance  in  criterion 
accounted  for 

df  =  Degrees  of  freedom  used  in  testing 
significance  of  R 
F  =  Resultant  F  value 

(2)  Verbal  Description: 

(a)  Acquisition:  The  main  effects  of  complexity, 
feedback  embeddedness,  and  trials  were  all  found 
to  influence  acquisition  performance,  as  expected. 
There  was  a  significant  interaction  between  task 
complexity  and  trial  block.  The  initial 
differences  in  error  rate  due  to  various  levels  of 
complexity,  although  maintained  across  trails, 
decreased  as  training  continued.  A  greater 
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average  number  of  errors  resultes  from  removal  of 
feedback,  even  though  fewer  responses  are  required 
in  such  tasks,  although  the  interaction  with  task 
complexity  was  not  significant.  As  the  level  of 
embedding  increases,  errors  become  a  function  of 
increasing  levels  of  feedback;  conversely,  as  the 
percentage  of  distracting  stimuli  decreased, 
increasing  errors  were  associated  with  decreasing 
feedback.  Feedback  had  a  simpler  and  more 
systematic  effect  on  performance  time;  the 
results  suggest  that  tasks  consisting  of  more 
responses  take  relatively  longer  to  perform  than 
tasks  consisting  of  fewer  responses. 

In  general,  even  when  the  effect  upon  performance 
time  due  to  the  number  of  responses  is  removed, 
significant  multiple  correlations  between  task 
indices  and  time  are  still  obtained  at  all  three 
acquisition  stages. 

(b)  Transfer:  The  main  effects  of  complexity, 
feedback  and  trial  block  were  found  to  impact  upon 
transfer  performance  as  expected.  Transfer  from 
the  more  complex  device  is  better  than  transfer 
from  the  less  complex  device,  given  that  the 
critical  feature  of  feedback  is  present.  Presence 
or  absence  of  feedback  during  training  has  its 
most  marked  effect  on  transfer  for  complex  tasks, 
its  smallest  effect  for  simple  tasks,  and  an 
intermediate  effect  for  the  medium  task.  These 
differences  tend  to  dimish  over  trial  block, 
although  they  are  not  still  prevelent  on  the  last 
transfer  trials. 

As  shown  in  the  table  above,  significant  multiple 
correlations  are  obtained  between  task  indices  and 
both  time  and  error  measures  at  each  stage  of 
transfe.r  The  differences  between  acquisition  and 
transfer  tasks  in  the  number  of  displays,  the 
percentage  of  controls  used,  and  the  weighted 
Display  Evaluative  Index  bear  strong  relationships 
to  the  criterion  at  each  point. 

Results:  EXPERIMENT  2 

Acquisition:  The  impact  of  task  complexity  on 

acquisition  criteria  was  similar  to  that  reported 
earlier  for  the  transfer  of  training  study. 

There  is  evidence  that  training  method  affects  the 
number  of  errors  made  during  acqusition.  The  most 
errors  occur  when  the  cold  panel  method  is  used. 
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The  hot-panel  and  pictorial  methods  are 
comparable.  Errors  as  a  function  of  the 
interaction  between  task  complexity  and  training 
method  approached  significance,  tending  to 
indicate  that  the  relative  inferiority  of  the  cold 
panel  approach  holds  only  for  the  complex  task 
situation.  Performance  time  was  not  influenced  by 
method  during  acquisition. 

Transfer:  Training  task  complexity  had  a  parallel 

effect  on  transfer  performance  to  that  in 
Experiment  1;  significantly  fewer  errors  occured 
following  training  on  a  more  complex  task  than  on 
a  simpler  one.  Time  scores  during  transfer  were  a 
function  of  an  interaction  between  acquisition 
task  complexity  and  trial  block. 

Training  method  consistently  interacted  with 
trails  to  determine  performance  during  transfer. 

A  significant  training  method  by  trials 
interaction  is  shown.  The  relative  superiority  of 
training  on  the  hot  panel  early  in  transfer 
decreases  over  time,  in  number  of  errors.  By  the 
end  of  the  transfer  period,  the  three  methods  are 
virtually  the  same  in  terms  of  error  rates.  A 
significant  training  method-  by-trial  interaction 
for  transfer  performance  time  was  found  for  the 
simple  task.  The  difference  in  per  ormance  time 
is  maintained  across  the  entire  transfer  period 
between  the  hot  panel  and  cold  panel  groups,  while 
the  pictorial  group,  after  an  initial  retardation 
relative  to  the  hot  panel  group,  rapidly  converges 
with  it. 

c.  Authors'  Conclusions:  Consistently  large  and 
intuitively  systematic  variations  in  performance  were 
obtained  as  a  function  of  task/trainer  configuration. 

The  predictive  power  of  the  indices  for  skill 
acquisition  was  upheld,  with  multiple  correlations 
substantially  the  same  as  found  in  Phase  II.  The 
pattern  of  predictors  changes  somewhat  in  Phase  III; 
for  example,  the  Display  Evaluative  Index  entered 
prominently  in  Phase  III  among  the  predictors  of  both 
time  and  error  scores;  it  did  not  appear  at  all  in 
Phase  II  analyses.  The  pattern  of  predictors  was  shown 
to  vary  across  criterion  mesures  and  across  time  blocks 
within  criterion  measures  during  acquisition.  Thus,  a 
simple  f igure-of-mer it  approach  to  device  evaluation 
was  not  supported  in  terms  of  acquisition  performance. 
The  transfer  data,  however,  show  greater  consistency, 
and  provide  greater  encouragement  for  a  f igure-of-mer it 
approach  when  transfer  of  training  criteria  was 
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employed. 

The  current  results  provide  an  instance  in  which  it  was 
possible  to  quantify  similarity  for  a  surrogate  "real 
world"  task  and  to  predict  performance  with  very  high 
validity.  High  validity  was  obtained  notwithstanding 
an  interaction  between  task  complexity  and  feedback. 

The  results  of  Experiment  2  provide  some  support  for 
the  hypothesis  of  an  interaction  between  task 
parameters  and  method  of  training.  During  acquisition, 
training  method  appeared  to  have  a  differential  effect 
for  the  complex  task,  with  cold  panel  generating  more 
errors  than  either  pictorial  or  hot  panel.  In  the 
transfer  data,  presence  or  absence  of  task  embeddedness 
generated  a  differential  performance  effect  for 
training  methods.  Dynamic  presentation  led  to 
consistently  faster  performance  across  transfer  blocks 
than  either  cold  or  pictorial  presentation.  Its 
superiority  was  greater  under  the  no-embedding 
condition.  Otherwise,  the  results  were  consistent  with 
those  of  earlier  studies.  An  initial  advantage  of 
dynamic  training,  upon  transfer,  rapidly 
dissipated — particularly  in  regard  to  the  pictorial 
methods. 

At  this  level,  the  indices  could  be  employed  as  one  of 
several  tools  to  support  the  training  expert's 
evaluation  of  alternative  prototype  devices — for 
example,  to  corroborate  or  question  judgements  already 
established  by  other  means.  More  rigousous  and 
confident  use,  however,  requires  cross  validation  on 
actual  training  devices. 
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1.  Authors:  Nataupsky,  Mark,  Waag, Wayne  L., 

Weyer,  Douglas  D. ,  McFadden,  Robert  W. ,  &  McDowell,  Edward 

2.  Title:  Platform  Motions  Contributions  Simulator 
Effectiveness;  Study  III  -  Interaction  of  Motion  with 
Field-of-View 

3.  Source:  AFHRL-TR-7 9-25 ,  November  1979 

4.  Topic  Keywords:  Flight  simulation  ;  Motion  simulation 
;  Motion  cues  ;  Fi eld-of-view  ;  Platform  motion  ; 

Transfer  of  training  .  5.  Short  Summary:  A  transfer  of 

training  study  of  the  interaction  between  flight  simulator 
motion  cuing  and  simulated  f ield-of-view  finds  no 
significant  interaction  between  the  two.  Furthermore,  no 
enhancement  of  transferred  skills  due  to  platform  motion 
itself,  was  found  in  the  training  of  flight-naive 
undergraduate  pilot  students  to  fly  the  T-37  jet  trainer. 

6.  Devices: 

a.  T-37  jet  trainer  (test  device  only) 

b.  Advanced  Simulator  for  Pilot  Training 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  Air  Force  Base,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
AFHRL,  Williams  Air  Force  Base,  AZ  85224 

8.  Experiment: 

a.  Number  of  Groups:  4 

b.  Description  of  Groups:  Subjects  -  each  group 
approximately  8  student  jet  pilots,  range  of  previous 
flight  experience  25-64  hours 

c.  Tests  or  Trials/Timing:  In-process  trials;  1  post 
test  ( checkf light) 

d.  Number  of  Different  Types  of  Measures:  3 

Description  of  Measurements  and  Ratings: 

(1)  Instructor  pilot-scored  special  data  card 
ratings  (simulator  and  aircraft) 

(2)  Quantified  data  from  simulator's  recording 
instruments 


e. 


(3)  Instructor  checkflight  ratings 


f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 


g.  Statistical  Methods:  ANOVA 


h. 


Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 
Four  conditions  for  ASPT  training: 

(a)  full  platform  motion  with  full 
f ield-of-view  (FOV) 

(b)  full  platform  motion  with  limited 
f ield-of-view 

(c)  no  platform  motion,  full  f ield-of-view 

(d)  no  platform  motion,  limited  f ield-of-view 

(2)  Fidelity  Levels: 


Motion,  full  FOV 
Motion,  Limited  FOV 
No  Motion,  full  FOV 
No  Motion,  Limited  FOV 


Physical 
H  igh 

Med.  High 
H  igh 

Med.  High 


Functional 
H  igh 
H  igh 

Med.  High 
Med.  High 


(3)  Type  of  Task/Skill  Required:  aircraft  pilot 
operations;  cognitive;  psychomotor;  perceptual; 
part-task 

(4)  Task  Difficulty:  Medium  high 

i.  Stage  of  Training:  introduction;  familiarization; 
skill;  transition 


j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  POI:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  freeze  capability; 
restart/resequence  capability;  number/qual ity  of 
responses;  record/playback 


Abstract : 


a.  Study  Synopsis:  This  study  was  designed  to 
determine  the  effects  of  motion  cueing,  f ield-of-view, 
and  their  interaction,  on  (1)  skill  acquisition  in  a 
flight  simulator  and  (2)  transfer  of  learning  to 
aircraft,  in  undergraduate  jet  pilot  trainees  with 
little  prior  flying  experience. 


The  device  used  for  training  was  the  Advanced  Simulator 
for  Pilot  Training,  a  high-fidelity  full  mission 
instrument  and  visual  flight  simulator  capable  of  full 
f ield-of-view  (FOV)  with  great  visual  detail,  and  of 
platform  motion  that  provides  translational  and 
rotational  onset  cues  to  the  student  pilots  with 
between  25  and  64  hours  prior  flying  experience.  They 
were  divided  into  four  groups  of  approximately  8  each 
and  separately  trained  on  specific,  uniform  maneuvers 
in  the  simulator  under  four  different  conditions:  (1) 
full  platform  motion  with  full  FOV;  (2)  full  platform 
motion  with  limited  FOV;  (3)  no  platform  motion  with 
full  FOV  and  (4)  no  platform  motion  with  limited  FOV. 
Subjects  were  subsequently  tested  for  transfer  to  a 
T-37  jet  training  aircraft  on  one  sortie,  and  scored  on 
the  same  maneuvers  they  had  run  in  the  simulator. 
Measures  taken  in  the  simulator  were  instructor  ratings 
and  quantified  data  from  the  simulator  recording 
instruments;  measures  on  the  aircraft  were  instructor 
ratings. 


b.  Results: 

(1)  Key  Data:  Mean  T-37  Evaluation  Data 
(Instructor  Pilot  Ratings) 


Task 

Motion 

FOV 

Motion 

i-on 

Motion 

-off 

On 

Off 

full 

lim. 

full 

lim. 

full 

lim. 

Takeoff 

3  .88 

2.56 

3.00 

3.44 

3.75 

4.00 

2.25 

2.88 

Steep  Turn 

3.06 

1.93 

2.50 

2.53 

3.50 

2.63 

1.50 

2.43 

Slow  Turn 

3.32 

2.27 

2.94 

2.67 

3.63 

3.00 

2.25 

2.29 

Straight-In 

2.44 

1.93 

2.00 

2.43 

2.38 

2.50 

1.63 

2.33 

(2)  Verbal  Description:  There  is  little  evidence 
that  motion,  f ield-of-view,  or  their  interaction 
significantly  affected  skill  acquisition  in  the 
simulator . 

Significant  performance  differences  did  occur, 
however,  among  the  groups  during  simulator 
training  for  the  motion  factor.  For  certain 
measures,  the  measured  trial,  including  the  first; 
although  the  underlying  reason  is  unclear. 

For  transfer  of  training  data  collected  in  the 
aircraft,  neither  motion,  f ield-of-view,  not  their 
interaction  during  simulator  training 
differentially  affected  performance  as  measured  by 
instructor  ratings.  There  was  a  trend  toward 
better  performance  by  the  motion-trained  groups. 

Of  the  58  statistical  tests  computed  on  the 
aircraft  evaluation  data,  only  three  produced 
significant  effects.  The  extent  to  which  these 


represent  real  effects  is  unknown,  since  the 
provability  of  significant  differences  given  the 
number  of  tests  is  quite  high. 

Due  to  Air  Training  Command  rules,  this  study 
could  not  include  a  control  group,  but  there  are 
good  reasons  to  assume  the  transfer  of  learning 
did  occur. 

c.  Authors'  Conclusions:  These  data  provide  support 
for  previous  findings  that  platform  motion  cueing  does 
not  significantly  enhance  the  transfer  of  learning  for 
basic  contact  tasks  in  the  T-37  aircraft. 

It  would  seem  that  the  impact  of  peripheral  visual  cues 
for  initial  acquisition  is  not  critical;  furthermore, 
no  convincing  evidence  was  found  indicating  increased 
transfer  using  platform  motion  in  conjunction  with  a 
narrow  f ield-of-view. 

It  seems  reasonable  to  conclude  that  no  substantial  or 
practical  differences  in  training  effectiveness 
resulted  from  manipulation  of  platform  motion  cueing 
and  the  f ield-of-view  of  the  visual  scene. 


1.  Authors:  Pohlmann,  Lawrence  D,  &  Reed,  John  C. 

2.  Title:  Air-to-Air  Combat  Skills:  Contribution  of 
Platform  Motion  to  Initial  Training 

3.  Source:  AFHRL-TR-7 8- 53 ,  October  1978 

4.  Topic  Keywords:  Flight  Simulation  ;  Motion  Cues  ; 

Air  Combat  Maneuvering  ;  Transfer  of  Training  ; 

Training  Effectiveness  . 

5.  Short  Summary:  No  significant  differences  in 
performance  in  real  aircraft  combat  training  was  found 
between  student  pilots  who  had  received  simulated  combat 
training  prior  to  aircraft  flight,  and  those  who  had  not. 

6.  Devices: 

a.  F-4  fighter  (test  device  only) 

b.  Simulator  for  Air-to-Air  Combat:  high  fidelity 
combat  flight  simulator 

(1)  With  6  degrees  of  freedom  platform  motion 

(2)  Without  motion 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
AFHRL,  Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  8  military  F-4  pilot  students 
matched  by  past  perf ormance--trained  in  simulator 
without  motion 

(2)  Subjects:  8  same,  trained  in  simulator  with 
platform  motion 

(3)  Controls:  8  same,  received  no  simulator 
training 

c.  Tests  or  trials/timing:  In-process  trials;  4 
post-test  (real  aircraft  check f 1 l ghts) 

d.  Number  of  different  types  of  measures:  2 
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e.  Description  of  measurements  and  ratings: 

(1)  Instructor  grading  for  each  trial  of  each 
maneuver 

(2)  Instructor  grading  overall  for  each  measure 

f.  Experimental  setting/training  context: 
institutional,  hands-on 

g.  Statistical  methods:  one-way  ANOV^ 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  above,  section  6 

(2)  Fidelity  Levels: 

Physical  Functional 

With  Motion  High  High 

No  Motion  High  Medium-High 

(3)  Type  of  task/skill  required:  aircraft  combat 
piloting;  operations,  cognitive,  psychomotor, 
perceptual,  part-task 

(4)  Task  difficulty:  high 

i.  Stage  of  training:  familiarization,  skill, 
transition 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude: 

(1)  Instructors:  good  for  their  own  use; 

doubtful  of  device  suitability  for  students 

(2)  Students:  positive 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  freeze  capability; 

restart/resequence  capability;  record/playback 
very  rarely  (feature  not  reliable);  others  not 
specif ied 

9.  Abstract: 

a.  Study  Synopsis:  This  study  was  designed  to 
determine  the  training  effectiveness  of  the  Simulator 
for  Air-to-Air  Combat  (SAAC)  in  teaching  skills  to 
student  pilots  in  the  initial  combat  phase  of  training, 
as  well  as  to  determine  of  6-degrees-of-f reedom 
platform  motion  contributed  to  learning  in  the  SAAC. 

Two  groups  of  8  students  pilots  each  were  trained  in 
the  SAAC,  one  with  motion,  one  without  motion.  A 
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third,  control  group,  received  no  simulator  training. 
The  three  groups  were  subsequently  tested  for  transfer 
in  four  real-aircraft  missions.  Instructors  graded 
students  on  the  execution  of  10  Basic  Fighting 
Maneuvers  and  on  9  discrete  skills  (e.g.,  "Range 
Estimation",  "Attitude"),  both  in  the  simulator  and  in 
the  aircraft. 

Both  students  and  instructors  also  completed 
questionnaires  on  their  simulator  training. 

Limitations  of  the  simulated  training  in  this 
experiment  were  noted:  (1)  the  record/playback  feature 

of  the  SAAC  was  very  rarely  used--it  was  unreliable  and 
scheduled  for  upgrading;  (2)  the  problem  freeze 
feature  was  rarely  used  due  to  instructor  reluctance; 

(3)  in  general,  instructors  were  reluctant  to  use  any 
device  feature  other  than  reset. 

A  further  technical  impediment  to  training  inherent  in 
the  simulator  design  was  that  instructors  "flew"  the 
simulated  missions  from  the  cockpit  of  the  adversary 
aircraft;  this  reduced  communication  between 
instructor  and  student. 

b.  Results: 


(1)  Key  Data: 

Mean  Performance  Ratings  for  Maneuvers 


Phase 

No-SAAC 

No-Motion 

Motion 

P 

Initial  Simulator 

- 

3  .94 

3  .93 

.  96 

Final  Simulator 

- 

4.85 

5  .45 

.  03 

Initial  Aircraft 

4.03 

3.73 

3  .64 

.  29 

Final  Aircraft 

4  .89 

4  .56 

4  .47 

.  06 

Overall  Aircraft 

4.44 

4.20 

4.13 

.  12 

Mean  Performance 

Ratings  for 

Skills 

No-SAAC 

No-Motion 

Motion 

P 

Initial  Simulator 

- 

5.65 

5.02 

.  05 

Final  Simulator 

- 

5.58 

6  .49 

.  01 

Initial  Aircraft 

5  .17 

4  .79 

4  .96 

.  50 

Final  Aircraft 

5.22 

5.24 

5  .19 

.  98 

Overall  Aircraft 

4.98 

5.12 

5.12 

.79 

(2)  Verbal  description:  It  was  found  that 
learning  took  place  in  the  simulator  and  in  the 
aircraft  for  all  groups.  However,  the  students 
trained  in  the  SAAC  did  not  perform  better  in  the 
aircraft  than  those  who  had  received  no  simulator 
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training--if  anything,  the  statistically 
nonsignificant  edge  went  to  the  no-simulator  group 
(tables  above).  The  only  significant  differences 
were  between  the  motion  and  no-motion  groups 
during  simulator  training,  with  the  motion  group 
scoring  better  on  final  simulator  maneuvers,  and 
on  initial  and  final  simulator  skills--but  their 
superiority  did  not  transfer  to  real-aircraft 
performance. 

On  questionnaires,  students'  consensus  was  that 
SAAC  training  helped  them  initially  to  picture 
maneuvers  better  in  the  aircraft;  but  there  was 
no  consensus  as  to  whether  it  helped  them  perform 
better.  Instructors  generally  agreed, 
subjectively,  that  they  saw  no  difference  between 
simulator-  trained  students  and 

non-simulator-trainod  students  in  the  aircraft. 

c.  Authors'  Conclusions:  The  data  did  not  reflect  any 
noticeable  increment  of  effectiveness  due  to  platform 
motion.  There  were  no  consistent  transfer  of  training 
differences  between  the  motion  and  no-motion  groups. 

If  anything,  there  is  some  indication  that  the  students 
who  did  not  receive  SAAC  training  performed  better  in 
the  aircraft. 

The  reader  should  keep  in  mind  the  integral 
relationship  of  training  program  and  training  device. 

There  are  several  explanations  for  the  lack  of 
significant  aircraft  performance  differences  between 
groups : 

(1)  Instructor  ratings  may  not  have  been 
sufficiently  sensitive  or  consistent 

(2)  "Performance"  grades  may  not  be  measuring  the 
kind  of  conceptual  knowledge  derived  from  the 
simulator . 

(3)  The  fact  that  simulator  training  in  this  study 
was  an  additional  activity,  over  and  above  other 
schooling,  may  have  impaired  its  effectiveness. 

(4)  Device  design,  with  the  instructor  in  a 
removed  cockpit  (the  adversary  aircraft)  may  not 
be  suited  to  the  initial  phase  of  learning  combat 
maneuver s--thi s  was  a  majority  opinion  of 
instructors. 

(5)  Important  instructional  features  of  the  device 
were  not  used,  or  rarely  used,  because  of  the 
reluctance  of  instructors  and,  in  the  case  of  the 
playback  feature,  because  the  equipment  was 
unreliable  and  due  for  upgrading. 
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The  results  of  the  current  experiment  leave  the 
question  of  the  need  for  platform  motion  for  training 
air-to-air  combat  tasks  unanswered. 
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f.  Experimental  Setting/Training  Context: 

Institutional,  hands-on 

g.  Statistical  Methods:  Chi  Square 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Ground  based  flight 
simulators,  Link  AN-T-18  and  GAT-1 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified,  presumably  medium 
high  to  high  for  both 

(b)  Functional:  presumably  high  for  GAT-1, 
medium-  high  for  Link  AN-T-18 

(3)  Type  of  Task:  Operations  (flight) 

(4)  Task  Difficulty:  High 

(5)  Skills  required  by  task:  motor,  perceptual, 
cognitive 

i.  Stage  of  Training:  Familiarization  and  Skill 

j.  Trainee  Sophistication:  Novice 

k.  Incorporation  of  Device  into  P.O.I.: 

Instructor-managed  individually 

»  » 

i.  User  Acceptance  or  Attitude: 

(1)  Instructors:  Good 

(2)  Students:  Not  discussed 

m.  Use  of  Instructional  Features:  Intensive 

I 

9 .  Abstract : 

a.  Study  Synopsis:  The  relative  benefits  of  different 
types  of  flight  training  equipment  were  evaluated  in  a 
routine  instructional  situation  with  no  particular 

constraints  placed  upon  the  instructors  as  to  how  they  t 

used  the  equipment,  and  without  interfering  with  the 
normal  course  of  flight  training.  The  objectives  of 
this  research  were: 

(1)  to  determine  the  relative  training  value  of  11 
hours  flight  instruction  in  two  different  ground 

trainers,  as  compared  with  in  the  aircraft;  » 

(2)  to  evaluate  the  flight  instructors'  ability  to 
predict  success  in  private  pilot  training  on  the 
basis  of  students'  initial  performance  in  each  of 
two  ground  trainers  as  opposed  to  actual  aircraft; 

(35  to  develop  an  objective  scale  for  checking 

flight  proficiency.  > 

The  ground  trainer  groups  passed  their  flight  checks 
with  an  average  of  slightly  more  than  an  hour  greater 
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1 .  Authors : 


Povenmire,  H.  Kingsley,  &  Roscoe,  Stanley  N. 

2.  Title:  An  Evaluation  of  Ground-Based  Flight  Trainers  in 
Routine  Primary  Flight  Training 

3.  Source:  Human  Factors  ,  1971,  13(2),  109-116 

4.  Topic  Keywords:  Flight  Training  ; 

Transfer  Effectiveness  ;  Ground-based  Simulator  ; 

Transfer  Effectiveness  Ratio  . 

5.  Short  Summary:  Ground-based  simulated  flight  training 
can  be  equally  as  effective  as  aircraft  flight  training  up 
to  some  cut-off  point  in  time  that  may  be  indicated  with  a 
simple  "Transfer  Effectiveness  Ratio". 

6.  Devices:  Ground-based  flight  simulators: 

a.  Link  AN-T-18  medium-high  fidelity 

b.  GAT-1  general  aviation  trainer  high  fidelity 

7 .  Institutions  : 

a.  Sponsor:  Link  Foundation 

b.  Performing  Organization:  Aviation  Research 
Laboratory,  Institute  of  Aviation, 

University  of  Illinois  ,  Savoy,  IL  61874 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Controls:  11  aviation  college  students 

without  previous  flight  experience 

(2)  Subjects:  13  same 

(3)  Subjects:  12  same 

c.  Measures  and  Timing  of  Measures:  Pre-evaluation, 
and  post-tests 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Pre-evaluation:  instructor  prediction  of 

performance  I 

(2)  Post-test:  score  on  final  check  flight 

(3)  Amount  of  training  time  required  to  reach  / 

final  check  flight 


> 
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trainer--theref ore,  cockpit  procedures  trainers 
should  be  used  prior  to  aircraft  training  if 
maximum  benefit  is  to  be  derived  from  their  use; 
(5)  there  is  substantial  task  identity  between 
ground  cockpit  procedures  in  the  aircraft  and  in 
the  two  simulation  devices  studied. 
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groups.  Both  groups  reached  an 
e  of  error-reduction  at  the  same  rate 
the  actual  aircraft.  Moreover,  the 
groups  took  less  training  time  to 
did  the  aircraft-trained  group. 


Emergency  procedures  were  not  covered  in  this  study, 
because  neither  simulator  was  deemed  to  approach  the 
stress  and  time-sharing  loads  that  exist  in  a  real 
aircraft  emergency. 

This  study  appears  to  confirm  that  task  fidelity,  which 
in  many  cases  can  be  achieved  in  a  simulator  of  low 
physical  fidelity,  is  as  important  as  or  more  important 
than  physical  fidelity  in  the  acquisition  and  transfer 
of  procedural  skills 

b.  Results: 

(1)  Key  Data: 

MEAN  PERCENTAGE  OF  ERROR  ON  AIRCRAFT 
(TRANSFER)  TRIALS  BY  TRAINING  CONDITION 

TRAINING  CONDITION 


Aircraft (N=10)  2-C-9(N=10) 

Mockup (N=10 ) 

Trial  1 

1  9.6 

6.1 

6.3 

Trial  2 

11.6 

4  .8 

3.7 

Trial  3 

7  .2 

2.9 

2.7 

Trial  4 

5  .5 

3.2 

2.1 

Trial  5 

4  .3 

3.8 

3.7 

Trial  6 

2  .3 

no  trial 

no  trial 

(2)  Ver 

bal  Description: 

Lower  error 

rates  for 

trainer 

groups  than  for 

ai rcraf t-trained  group 

significant  at  d  .001. 

Differences 

between  two 

trainer 

groups  not  signi 

f icant. 

Authors ' 

Conclusions : 

(1)  The 

2-C-9  trainer  (h 

igh  fidelity)  is  an 

ef  f  ecti' 

ve  device  for  tea 

ching  pre-start,  start, 

run-up, 

and  shut-down  pr 

ocedures  for 

the  OV-1 

aircraft; 

( 2 )  the 

mockup  trainer  i 

s  an  equally 

effective 

device  for  teaching  these  ground  procedures; 

(3)  groung  cockpit  procedures  can  be  taught  quite 
effectively  in  devices  of  low  physical  fidelity 
which  are  quite  inexpensive  to  fabricate; 

(4)  ground  cockpit  procedures  can  also  be  learned 
rapidly  in  the  aircraft  without  benefit  of  a 
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f.  Experimental  Setting/Training  Context:  Voluntary 
experiment 

g.  Statistical  Methods:  Not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  sophisticated  computerized  cockpit 
procedures  trainer  2-C-9; 

(b)  rudimentary  mockup  of  aircraft  cockpit 
made  of  plywood  and  photographs 

(2)  Fidelity  Levels:  High  and  Low 

(3)  Type  of  Task/Skill  Required:  Ground  cockpit 

procedures:  174  items 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  Experimental 

j.  Trainee  Sophistication:  High;  all  had  minimum  291 
hours,  maximum  4,962  hours  flight  experience 

k.  Incorporation  of  Device  into  P.O.I.: 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  Not  discussed 

(2)  Students:  Not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  Not  discussed 

(2)  Features  used:  Not  discussed 

9.  Abstract: 

a.  Study  Synopsis:  It  has  been  shown  that  transfer  of 
training  from  a  simulator  to  actual  equipment  does  not 
necessarily  vary  in  proportion  to  the  "face  validity" 
or  physical  fidelity  of  the  simulator.  The  evidence 
indicates  that  task  fidelity  is  more  important  than 
physical  fidelity  in  the  transfer  of  training.  This 
study  compared  the  effectiveness  of  a  low-fidelity, 
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1.  Authors:  Prophet,  Wallace  W. ,  &  Boyd,  H.  Alton 


2.  Title:  Device-Task  Fidelity  and  Transfer  of  Training: 
Aircraft  Cockpit  Procedures  Training 

3.  Source:  HumRRO  Technical  Report  70-10,  July  1970 

4.  Topic  Keywords:  Simulated  Flight  Training  ; 

Transfer  of  Training  ;  Low  Fidelity  . 

5.  Short  Summary:  No  significant  differences  in  training 
effectiveness  were  found  between  cockpit  procedures  training 
devices  of  high  and  low  physical  fidelity,  as  measured  by 
subsequent  trials  in  actual  aircraft.  This  suggests  that 
task  fidelity  is  more  important  than  physical  fidelity  in 
the  transfer  of  procedural  training. 

6 .  Devices : 

a.  Sophisticated  computerized  cockpit  trainer 

b.  Inexpensive  mock-up  of  aircraft  cockpit 

7.  Institutions: 

a.  Sponsor:  Office,  Chief  of  Research  and 
Development,  Department  of  the  Army 

b.  Performing  Organization:  HumRRO  Div.  No.  6 
(Aviation),  Fort  Rucker,  AL  36362 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  Two  Group  Post  Test 

b.  Description  of  Groups: 

(1)  Subjects:  20  non-student  volunteers,  rated 
Army  aviators 

(2)  Controls:  10  same 

c.  Tests  or  Trials/Timing:  post-test  flight  in  actual 
aircraft 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings: 

(1)  Performance  Measures:  Instructor-scored 
checklist  trials  in  actual  aircraft,  scored  by 

(a)  fraction  of  errors; 

(b)  time  to  accomplish 

(2)  Supervisor  Ratings:  None  applicable 


transfer  task;  only  in  Experiment  I,  however, 
where  experienced  pilots  were  the  subjects,  did 
the  T&E  group  also  perform  better  (but  only 
marginally)  during  the  training  trials;  in 
Experiments  II  and  III,  learning  curves  showed 
little  difference  between  groups. 

In  all  experiments,  stress  (electric  shocks) 
significantly  degraded  performance  for  all  groups, 
but  there  was  no  significant  difference  between 
groups  except  in  Experiment  I  (again,  the 
experienced  pilots),  where  the  T&E  group's 
performance  suffered  less  than  the  prompted 
group' s . 

c.  Authors'  Conclusions:  From  the  first  three 
experiments,  it  appears  that  during  training,  each 
method  chosen  is  equally  efficient;  but  after 
training,  tr ial-and-er ror  methods  assert  their 
superiority  in  the  transfer  task.  The  overall  evidence 
from  these  experiments  indicates  that  tr ial-and-er ror 
is  a  superior  method  of  training  when  compared  with 
highly  prompted  procedures  if  the  task  is  one  of 
difficult  perceptual  learning  such  as  the  one  involved 
here.  The  indiscriminate  use  of  prompting  techniques 
for  teaching  complex  tasks  is  open  to  question. 

From  the  fourth  experiment,  it  appears  that  it  is 
possible  to  train  in  resistance  to  stress  by  placing 
the  trainee  under  a  low  to  moderate  amount  of  stress 
during  the  training  program.  The  stress  trained 
student  should  then  perform  better  when  under  a  higher 
stress  condition  than  a  student  who  has  not  been  placed 
under  the  stressful  training  condition. 
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(1)  Training  conditions  with  training  target 

(a)  Trial-and-error  with  feedback 

(b)  Prompted  (errorless)  without  feedback 

(c)  Initially  prompted  without  feedback,  but 
after  3  trials,  trained  via  trial-and-error 
with  feedback 

(2)  Testing  conditions  with  either  training  or 
"transfer"  target  (a  different  picture) 

(a)  No  feedback 

(b)  Stress  condition:  small  electric  shock 
administered  if  in  error  (no  other  feedback) 

Somewhat  different  was  Experiment  IV  on  the  same  task, 
which  tested  the  hypothesis  that  subjects  who  were 
trained  under  moderate  stress  might  perform  better  in  a 
stressful  transfer  task  than  those  who  were  trained 
with  little  stress.  In  this  experiment,  during 
training  one  group  received  positive  feedback  on 
correct  execution--  the  word  "good"  from  the 
experimenter — while  the  other  group  received  negative 
feedback,  when  in  error,  in  the  form  of  a  small 
electric  shock.  The  stress  test  condition,  which  was 
represented  by  a  single  trial  at  the  end  of  the 
testing,  threatened  subjects  with  a  substantial  (200 
volt)  electric  shock  for  error. 

b.  Results: 

(1)  Key  Data: 

(a)  From  experiments  I,  II,  and  III; 

Transfer  trials  only: 

Experiment  I:  trial-and-error  group  mean 
error  664  ft. ;  prompted  group  mean  error 
1,115  ft.,  significant  at  .05  level 
Experiment  II:  T&E  group  mean  error  921  ft.; 
prompted  group  1,211  ft.  (p  <  .05) 

Experiment  III:  T&E  group  mean  error  846 
ft.;  combined  prompted-and  T&E  group  1,000 
ft.  (t  =  1.59,  p  .1) 

(b)  From  experiment  IV:  No  significant 
difference  on  transfer  trials  between  groups, 
except  on  the  single  stress  transfer  trial, 
the  positively  reinforced  group's  mean  error 
was  532  ft.  and  the  negatively  reinforced 
group's  was  414  ft.  (t  =  2.21,  p  .05) 

(2)  Verbal  Description:  Experiments  II  and  III 
confirmed  findings  of  Experiment  I,  that,  in 
agreement  with  most  of  the  literature,  T&E  groups 
performed  better  than  prompted  groups  on  a 
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e.  Description  of  Measurements  and  Ratings: 

(1)  Objectively  measured  deviation  from  correct 
estimation  of  range 

(2)  Heart-rate  of  subjects  (to  verify  stress) 

f.  Experimental  Setting/Training  Context:  laboratory, 
hands-on 

g.  Statistical  Methods: 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices,  as  above  section  6.  BY 

METHOD : 

(a)  With  or  without  cue  light  (for 
"error-free"  prompted  learning) 

(b)  With  or  without  electric  shock  (for 
supplying  stress  in  Experiments  I-III;  and 
for  supplying  negative  learning 
reinforcement,  Experiment  IV) 

(2)  Fidelity  Levels: 

(a)  Physical:  Low 

(b)  Functional:  Low 

(3)  Type  of  Task/Skill  Required:  Simulated 
firing;  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  skill 

j.  Trainee  Sophistication:  Experiment  I, 
intermediate;  Experiment  II,  novice 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 
( exper imental ) 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  augmented  feedback 

9 .  Abstract : 

a.  Study  Synopsis:  This  paper  discusses  four 
experiments  evaluating  different  training  techniques  in 
teaching  a  moderately  complex  perceptual-psychomotor 
task.  The  task  under  study  was  range-estimation  as 
measured  by  the  accuracy  of  initiation  of  fire  on  a 
stationary  target  in  a  simulated  strafing  run.  In 
experiments  I  through  III,  several  tr ial-and-er ror 
versus  highly  prompted  (highly-  cued,  errorless) 
training  and  testing  conditions  were  applied,  as 
follows : 
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1.  Authors:  Prather,  Dirk  C.  ,  Berry,  Gene  A.,  & 

Jones,  Gerald  L. 

2.  Title:  Four  In-Depth  Studies  of  a  Flight  Training 

Skill:  Trial  and  Error  versus  Prompted  Learning  Evaluated 

on  Efficiency,  Transfer,  and  Stress 

3.  Source:  U.S.  Air  Force  Research  Report  71-11,  December 
1971 

4.  Topic  Keywords:  Range  estimation  task  ; 

Tr ial-and-er ror  training  ;  Errorless  training  ; 

Prompted  learning  ;  Stress  . 

5.  Short  Summary:  Comparison  of  tr ial-and-er ror  versus 
prompted  learning  techniques  in  training  for  a  complex 
perceptual-psychomotor  skill  finds  the  tr ial-and-er ror 
technique  superior  when  transferred  to  a  similar  task. 

6.  Devices:  Scaled  two-dimensional  projection  of  strafing 
run  area  target,  and  trigger  for  simulated  firing. 

Projected  picture  differed  between  training  and  transfer. 

7.  Institutions: 

a.  Sponsor:  Dean  of  Faculty,  U.S.  Air  Force  Academy, 
CO  80840 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment  (4  experiments) 

a.  Number  of  Groups:  Experiment  I,  6;  Experiment  II, 
4;  Experiment  III,  2;  Experiment  IV,  2 

b.  Description  of  Groups: 

EXPERIMENT  I:  ( 1 ) — ( 6 )  16  each  group,  moderately 

experienced  male  student  pilots,  randomly  assigned 

EXPERIMENT  II:  ( 1 ) — ( 4 )  15  each  group,  inexperienced 

male  student  pilots,  randomly  assigned 

EXPERIMENT  III:  ( 1 ) — ( 2 )  20  each  group,  same  as  Exp. 

II 

EXPERIMENT  IV:  ( 1 ) - ( 2)  20  each  group,  same  as  Exp.  II 

c.  Tests  or  Trials/Timing:  in-process  trials; 
post-tests 

d.  Number  of  Different  Types  of  Measures:  2 
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error  method.  The  mean  of  the  tr ial-and-er ror  group 
was  16%  better  on  performance  than  the  mean  of  the 
errorless  group,  although  the  difference  could  not  be 
considered  (more  than  marginally)  significant  due  to 
the  large  amount  of  variance  in  performance  because  of 
the  low  number  of  training  trials. 

The  results  on  efficiency  of  training  may  be  contrary 
to  the  literature  because  of  the  difficulty  of  the 
training.  Possible  there  is  a  point  in  the  difficulty 
of  discr imnination  tasks  at  which  trial-and-  error 
methods  overtake  and  pass  errorless  procedures  in 
efficiency.  Another  reason  for  the  results  may  be  that 
most  adults  have  a  long  history  of  tr ial-and-error 
learning;  they  probably  have  learned  to  be  efficient 
at  this  process.... 

In  agreement  with  most  of  the  literature,  however, 
tr ial-and-error  training  was  superior  to  errorless 
training  when  teaching  for  transfer.  These  results 
extend  the  literature  to  a  more  difficult  perceptual 
task . 


Answers  were  supplied  by  depressing  a  trigger.  The 
projected  target  in  training  trials  was  a  conventional 
cloth  training  target.  During  training, 
trial-and-error  subjects  were  given  feedback  on  the 
accuracy  of  their  responses--by  the  experimenter's 
verbal  statement  of  actual  distance  at  which  they  had 
fired,  immediately  after  every  firing.  Error-free 
subjects  were  required  to  fire  at  the  appearance  of  a 
cue  light  which  came  on  at  the  correct  approach 
distance. 

After  the  first  set  of  training  trials,  both  groups 
were  given  trials  without  feedback  and  without  cuing 
lights  for  either.  After  training  and  trials  with  the 
cloth  target,  the  six  groups  were  tested  with  the  same 
device  in  either  of  three  ways: 

(1)  "Transfer":  the  projected  target  was  changed 
to  a  MIG-21? 

(2)  "Stress":  the  target  was  the  cloth  practice 
target  but  subjects  received  electric  shocks  for 
incorrect  responses 

(3)  "Transf er-with-Str ess " :  target  was  MIG-21  and 
shock  was  administered  for  incorrect  responses. 

b.  Results: 

(1)  Key  Data: 

MEAN  ERROR  IN  FEET  BY  TRAINING  CONDITION 

Training  Transfer  Stress  Transfer  plus 
Condition  Test  Test  Stress  Test 

Trial-and-error  664  427  1,080 

Errorless  1  ,115  7  50  1,218 

P ■  0.05  P' 0.01  not  significant 

(2)  Verbal  Description:  The  trial-and-error  group 
showed  a  marginally  significant  superior 
performance  on  the  learning  trials  when  neither 
group  was  given  feedback  or  cues. 

The  trial-and-error  group  performed  with 
significant  and  larger  superiority  on  the  transfer 
test  and  on  the  stress  test.  There  was  no 
significant  difference  on  the  transf er-with-stress 
tests. 

c.  Author's  Conclusions:  Contrary  to  most  of  the 
literature,  the  errorless  method  did  not  teach  the 
experimental  task  more  efficiently  than  the  trial-and- 


e.  Description  of  Measurements  and  Ratings: 

Objectively  measured  deviation  from  correct  estimate  of 
target  range. 

f.  Experimental  Setting/Training  Context:  Laboratory 
experiment,  hands-on 

g.  Statistical  Methods:  One-way  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Two-dimensional  projection 
of  strafing  run  area  and  target 

(a)  With  or  without  cue  light  (for  error-free 
cuing  of  response; 

(b)  with  or  without  electric  shock  to  wrist 
(for  stress) 

(c)  projected  either  picture  of  cloth  target 
for  trials;  or  picture  of  MIG-21,  for 
transfer 

(2)  Fidelity  Levels: 

(a)  Physical:  Low 

(b)  Functional:  Medium 

(3)  Type  of  Task/Skill  Required:  motor; 
perceptual;  part-task  (very  narrowly-defined 
task) 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  skill 

j.  Trainee  Sophistication:  intermediate  and  expert 

k.  Incorporation  of  Device  into  P.O.I.:  not 
appl icable 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  specified 

(2)  Students:  not  specified 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  This  experiment  attempted  to 
evaluate  an  error-free  training  method  wherein 
error-commission  was  practically  impossible  and  no 
feedback  on  performance  was  given  to  the 
er ror-f ree-trained  subjects.  The  task  involved  a 
visual  estimate  of  correct  distance  from  a  target  to  be 
fired  on  at  the  optimum  point  in  a  strafing  run 
approach,  as  represented  by  a  movie  film  projection. 


1.  Author:  Prather,  Dirk  C 
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2.  Title:  Tr ial-and-Er r or  Versus  Errorless  Learning: 
Training,  Transfer,  and  Stress 

3.  Source:  American  Journal  of  Psychology,  1971,  Vol.  84, 
No.  3 

4.  Topic  Keywords:  Range-estimation-task  ; 

Tr ial-and-er ror  learning  ;  Errorless  learning  ; 

Error-free  learning  ;  Stress  . 


5.  Short  Summa.y:  A  comparison  of  training  methods  for 
teaching  a  transferable  perceptual  skill  finds  the  trial- 
and-error  method  superior  to  an  error-free  method. 

6.  Device:  Two-dimensional  projection  of  strafing-run  area 
and  target;  trigger  for  simulated  firing; 
shock-administering  device;  and  cue  light 


7.  Institutions: 

a.  Sponsor:  not  named;  some  agency  of  U.S.  Air 
Force 

b.  Performing  Organization:  not  named;  some  agency 
of  Arizona 

8.  Type  of  Article:  Experiment 


a.  Number  of  Groups:  6 

b.  Description  of  Groups: 


(1) 

Sub j  ects 

:  tr 

ained  by 

er  r 

or-free  method 

(a)  16  male 
Transfer 

students 

at 

Luke  AFB ,  tested  for 

(b) 

16 

same , 

tested 

for 

Stress 

(c) 

16 

same , 

tested 

for 

Transfer-wit h- Stress 

(2) 

S  ub  j 

ects 

:  tr 

ained  by 

tr  i 

al-and-error  method 

(a) 

16 

same , 

tested 

for 

Transfer 

(b) 

16 

same , 

tested 

for 

Stress 

(c) 

16 

same , 

tested 

for 

Tr an sfer-with- Stress 

NOTE:  All  subjects  randomly  assigned  from  a  pool 

of  96  men  with  age  range  21-44,  mean  age  25.9 
years.  All  moderately-experienced  pilots  who  had 
received  wirgs  in  a  USAF  flight  training  program. 

c.  Tests  or  Trial s/Timing:  In-Process  Trials; 

Immediate  Post-Tests 

d.  Number  of  Different  Types  of  Measures:  I 
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total  time  than  those  trained  exclusively  in  the 
aircraft.  A  Transfer  Effectiveness  Ratio  was  developed 
to  show  the  relative  transfer  effectiveness  of 
simulator  training  (see  "Authors'  Conclusions"). 

A  correlation  between  instructors'  predictions  and 
subsequent  student  performance  indicated  that  initial 
evaluations  made  in  the  simulators  were  better  guides 
to  eventual  performance  than  evaluations  made  in  the 
aircraft. 

The  third  objective  was  answered  by  quantitative 
descriptions  of  performance  criteria  in  maneuvers 
proposed  by  the  15  flight  instructors.  These 
descriptions  showed  possiblilities  for  developing 
objective,  quantifiable  performance  criteria  that  could 
be  specified  through  further  flight  experimentation. 


b.  Results: 

(1)  Key  Data: 

Hours  to 

Checkflight 

Final 

Ground 

Score 

Ai rcr af  t 

Trainer 

Total 

Aircraft-trained 
Students  (N=il)  4.07 

47.60 

0 

47.60 

AN-T-1 8- trained 

Students  (N=13)  4.01 

36.90 

10.73 

47.63 

GAT-l-trained 

Students  ( N= 12)  4.13 

36.99 

1  0. 89 

47.88 

(2)  Verbal  Description: 

On  final 

flight 

checks, 

students  who  received  approximately  11  hours 
ground-  based  simulator  training  time  in  place  of 
aircraft  training  time  performed  as  well  as  those 
who  received  all  their  training  in  the  aircraft. 
Moreover,  there  was  a  positive  correlation  between 
hour s-to-checkf light  time  and  instructor 
predictions  of  hour s-to-checkf light  time  based  on 
pre-experiment,  observation  of  students  in  the 
simulators,  whereas  there  was  not  a  positive 
correlation  between  hours-  to-fina1  checkflight 
time  and  instructor  predictions  based  on 
observations  of  students  in  the  aircraft. 

c.  Authors'  Conclusions:  A  "Transfer  Effectiveness 

Ratio"  expressing  the  transfer  effectiveness  of 
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simulator  training  versus  aircraft  training  can  be 
determined  by  a  simple  computation: 

TER  =  (Yc  -  Ye)  /  Xe 

Where  Yc  =  flight  hours  required  by  control  group  to 
reach  some  criterion  of  proficiency  in  the  aircraft; 

Ye  =  corresponding  value  for  exper iemental  group; 

Xe  =  ground  trainer  hours  received  by  experimental 
group. 

At  some  point  the  incremental  transfer  effectiveness 
ratio  for  an  additional  hours  of  ground  trainer  time 
will  fall  below  the  trainer/aircraft  cost  ratio.  At 
this  point  it  will  become  unprofitable  to  give  further 
ground  training  in  the  same  flight  tasks. 

Ground-based  flight  trainers,  as  represented  by  the  new 
GAT-1  and  the  old  Blue  Box,  can  yield  high  transfer 
when  used  by  flight  instructors  who  are  left  to  their 
own  devices  in  a  routine  training  program. 

The  prediction  of  pilot  aptitude  based  on  a  student's 
performance  during  his  first  two  hours  in  a 
ground-based  trainer  appears  promising. 
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1.  Author:  Puig,  Joseph  A. 

2.  Title:  The  Training  Effectiveness  of  Device  3A105, 
Tracked  Vehicle  Driving  Trainer  (M48A3) 

3.  Source:  Technical  Note:  NAVTRAEQUIPCEN  TN-36,  November 
1972 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Tank  Driving  Simulator  ;  Tracked  Vehicle  Driving  Simulator 

5.  Short  Summary:  A  transfer-of- training  study  of  a 
tracked  vehicle  driving  simulator.  Device  3A105,  found  it 
cost-efficient  (at  a  cost  of  $2.10  per  hour)  in  training 
basic  tank  driving  tasks  in  comparison  with  the  actual  tank 
(at  a  cost  of  $6.05  per  hour).  There  were  no  significant 
performance  differences  found  between  the  two  groups. 

6 .  Devices : 

a.  M48A3  medium  weight  tank 

b.  Device  3A105  Tracked  Vehicle  Driving  Simulator 
(instrument  driving;  analog  computer  driven) 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization: 

Tracked  Vehicle  School,  Marine 
Corps  Base,  Camp  Pendleton,  CA  92055 

8.  Type  of  Article:  Experiment 

a.  Number  of  groups:  2 

b.  Description  of  groups: 

(1)  Subjects:  19  enlisted  men  and  3  officers, 
students  at  Marine  Tracked  Vehicle  Driving  School 

(2)  Controls:  12  enlisted  men  and  3  officers, 
same 

c.  .  Tests  or  tr  ial s/t  iming  :  post-tests 

d.  Number  of  different  types  of  measures:  4 

e.  Description  of  measurements  and  ratings: 

(1)  Instructor  ratings 

(2)  Time  to  proficiency,  practice  driving  time, 


instructor  time 

(3)  Number  of  instructor  corrections 

(4)  Written  examination  scores 

f.  Experimental  setting/training  context: 
institutional,  hands-on 

g.  Statistical  methods:  ANOVA 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  above,  section  6 

(2)  Fidelity  levels  (of  simulator): 

(a)  Physical:  Medium 

(b)  Functional:  Medium 

(3)  Type  of  task/skill  required:  tracked  vehicle 
driving;  operations,  cognitive,  psychomotor, 
perceptual,  whole-task 

(4)  Task  difficulty:  Medium-high 

i.  Stage  of  training:  introduction;  skill 
3.  Trainee  sophistication:  not  specified 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  instructional  features: 

(1)  Intensity:  not  specified 

(2)  Features  used:  Malfunction  selection,  others 
not  specified 

Abstract : 

a.  Study  Synopsis:  A  transf er-of- training  study  was 
designed  to  compare  the  effectiveness  of  equal  amounts 
of  training  time  in  an  actual  tank  and  in  a  tracked 
vehicle  driving  simulator.  The  experimental  group  of 
22  student  drivers  received  training  time  in  the 
simulator  equal  to  the  tank  training  time  received  by 
the  control  group  of  15  student  drivers.  Both  groups 
were  tested  by  checkride  in  an  actual  tank  on  eight 
separate  items,  ranging  from  gear  selection  to  moving 
into  and  out  of  deep  ditches,  and  ground  guiding  by  day 
and  night. 

An  analysis  of  per-hour  costs  of  tank  training  and 
simulator  training  was  also  done. 


b.  Results: 

(1)  Key  Data: 

Enlisted  Class,  January  '71,  Tank  Driver 
Training  Ratings  Scales,  Items  2-9, 
Unweighted  Means 

Tank-trained  S imulator- trained 
Total  Score  26.6  27.8 

(2)  Verbal  Description:  The  totals  of  the 
instructor  scores  on  the  eight  items  on  the 
checkrides  for  the  two  groups  is  shown  above. 

This  difference  was  not  statistically  significant, 
nor  were  there  significant  differences  on  any  of 
the  individual  items.  Other  performance  measures, 
such  as  number  of  instructor  corrections,  also 
showed  no  significant  differences. 

The  cost  of  training  per  hour  in  the  trainer  is 
$2.10;  in  the  tank,  $6.05. 

Q  •  c.  Author's  Conclusions:  Since  the  training  time 

required  by  the  tank  group  and  trainer  group  is 
approximately  the  same,  it  is  concluded  that  there  is  a 
one-to-one  substitution  ratio  between  the  trainer  and 
the  vehicle  itself  insofar  as  driver  training  is 
concerned.  In  addition,  since  there  is  a  significant 
cost  difference  between  the  two  methods  of  training, 
the  trainer  represents  a  more  cost-effective  method  of 
training  tank  drivers.  It  is  even  more  cost-effective 
when  considering  the  savings  in  transmission  repair 
costs  by  using  the  simulator.  Novice  drivers  have  been 
responsible  for  breakage  of  transmission  gear  trains 
while  practicing  in  the  actual  vehicle. 
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1.  Authors:  Randle,  Robert  J.  Jr.,  Tanner,  Trieve  A., 
Hamerman,  Joy  A.,  &  Showalter,  Thomas  H. 

2.  Title:  The  Use  of  Total  Simulator  Training  in 
Transitioning  Air-Carrier  Pilots:  A  Field  Evaluation 

3.  Source:  NASA  Technical  Memorandum  81250,  January  1981 

4.  Topic  Keywords:  Transition  Training  ;  Field  Evaluation 
;  Flight  Simulator  ;  Landing  Maneuver  . 

5.  Short  Summary:  A  study  of  checkflight  performances  of 
experienced  commerical  pilots  transitioning  to  B727  and  DC10 
aircraft  showed  no  significant  differences  relative  to 
whether  they  were  trained  completely  on  simulators  or  mostly 
in  the  aircraft. 

6.  Devices:  High  Fidelity  Visual  and  Instrument  Flight 
Simulators  for  B727  and  DC10 

7.  Institutions: 

a.  Sponsor:  United  Airlines 

b.  Performing  Organization: 

Ames  Research  Center,  NASA 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  4 

b.  Description  of  Groups: 


(1) 

Subjects  for 

B727 

Training : 

28 

experienced 

commercial  pilots 

(2) 

Controls  for 

B727 

Training : 

20 

same 

(3) 

Subjects  for 

DC1  0 

Training : 

26 

same 

(4) 

Controls  for 

DC1  0 

Training : 

20 

same 

Tests 

or  Trials/T 

iming 

one  post- 

tes 

t 

(checkflight) 

d.  Number  of  Different  Types  of  Measures:  4 

e.  Description  of  Measurements  and  Ratings: 

(1)  Instrument  Data: 

(a)  Descent  rate 

(b)  Descent  path  deviation 

(c)  Sink  rate  at  touchdown 

(d)  Vertical  acceleration  at  touchdown 

(e)  Lateral  acceleration  at  touchdown 

(f)  Standard  deviation  of  vertical 
acceleration  at  touchdown  plus  2  sec. 

[PREVIOUS  PAGE 
IS  BLANK 
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(g)  Standard  deviation  of  lateral 
acceleration  at  touchdown  plus  2  sec. 

(2)  NASA-observer  objective  and  rating-scale 

(3)  Checkpilot  rating  scale 

(4)  Trainee  ratings  of  training 

f.  Experimental  Setting/Training  Context:  NASA  Field 
Study  (as  described  by  experimenters)  at  United 
Airlines  Flight  Training  Center 

g.  Statistical  Methods:  Multivariate  analysis: 
discriminant  analysis;  principal-components  analysis 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  High  fidelity  visual  and 
instrument  flight  simulators  for  B727  and  DC10 

(2)  Fidelity  Levels: 

(a)  Physical:  high 

(b)  Functional:  high 

(3)  Type  of  Task/Skill  Required:  landing 
maneuvers 

(4)  Task  Difficulty:  high 

(5)  Skills  required  by  task:  motor,  perceptual, 
cognitive 

i.  Stage  of  Training:  transition 

j.  Trainee  Sophistication:  expert 

k.  Incorporation  of  Device  into  P.O.I.:  unspecified 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  discussed 

(2)  Students:  mixed  but  generally  good 

m.  Use  of  Instructional  Features:  intensive 
Abstract : 

a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
evaluate  a  transition  training  program  that  totally 
replaced  the  airplane  with  a  state-of-the-art  flight 
simulator.  The  evaluation  procedure  involved  analysis 
of  various  objective  measures  and  subjective  ratings  ci 
pilot  performance  as  a  step  toward  objectifying  and 
standardizing  assessment  techniques. 

135  experienced  pilots  transitioning  to  B727's  and 
DClO's  at  the  United  Airlines  Flight  Training  Center 
were  divided  into  two  pairs  of  groups,  of  which  ail 
completed  initial  normal  ground  school  and  normal 
initial  simulator  training.  Thereafter  the  control 
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groups  (one  each  for  each  aircraft)  progressed  to 
training  in  the  aircraft  (as  normal);  the  experimental 
groups  (one  for  each  aircraft)  received  all  their 
training,  prior  to  checkflight,  in  the  simulators. 

Evaluation  of  19  different  performance  measures  from 
five  sets  of  data  (NASA  observer  ratings,  checkpilot 
ratings,  trainee  ratings,  instrument-measured  inflight 
data,  instrument-measured  simulator  data)  showed  no 
statistically  significant  differences  between  the 
groups  on  their  performance  on  a  checkflight  which 
duplicated  standard  FAA  checkflight  procedures.  The 
small  (though  statistically  not  significant) 
superiority  of  the  ai rcraf t- trained  pilots  could  easily 
be  attributed  to  the  experimental  state  of  the 
simulator  training  program,  and  point  to  the  need  for 
work  on  the  simulator  training  curriculum. 


Comments  by  trainees  identified  specific  deficiencies 
in  simulator  realism,  particularly  in  t j he  perception 
of  depth,  and  of  sink  rate.  However,  there  was  a  high 
and  statistically  reliable  correlation  between  sink 
rates  in  the  simulator  and  in  the  aircraft,  which 
indicates  that  individual  performance  differences  could 
be  discerned  in  the  simulator.  This  implies  a  possible 
capability  for  predicting  aircraft  performance  from 
simulator  data  using  a  multiple  regression  equation. 
Furthermore,  precise  and  detailed  data  from  this  study 
comparing  performance  in  the  simulator  with  performance 
by  the  same  pilots  in  the  aircraft  may  provide  insight 
into  the  differences  and  similarities  between  simulator 
and  aircraft  in  respect  to  sink  rate  and  other 
phenomena . 

b.  Results: 


(  1 )  Key  Data: 


Phase  I 


Study  Results  by 


Aircraft  Type 


B- 

727 

DC-1 

0 

No. 

% 

No. 

% 

Sim 

Pass 

19 

68 

18 

69 

Fail 

9 

3  2 

8 

31 

A/C 

Pass 

18 

90 

2.0 

100 

Fail 

2 

10 

0 

0 

Note : 

The  airc 

raft 

-trained 

groups  se 

rved 

control  groups. 

(2)  Verbal  Description:  On  ail  measures  a 
significant  overlap  in  perfornance  between 
simulator-trained  and  aircraft-trained  pilots 
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obvious....  On  the  basis  of  these  performance 
measures  rt  was  not  possible  to  determine 
statistically  which  type  of  training  the  pilots 
had  experienced.  On  most  measures  there  was  a 
small  difference  in  favor  of  the  ai rcraf t- trained 
group.  But  the  differences  were  not  statistically 
significant.  Both  training  groups  improved  over 
the  three  landings  of  the  check  ride. 

High  sink  rates  in  the  simulator  were  associated 
with  high  sink  rates  in  the  aircraft  (and  low  with 
low).  The  correlation  was  high  and  statistically 
reliable. 

c.  Authors'  Conclusions:  The  two  training  groups  were 
seen  to  be  statistically  indistinguishable  on  the  basis 
of  the  many  performance  indices  that  were  utilized. 
Given  the  very  high  experience  level  of  the 
transitioning  pilots,  there  is  probably  such  a  swift 
transfer  of  previously  learned  skills  that  any  real 
differences  have  evaporated  before  they  can  be  measures 
by  the  techniques  used  in  this  study. 

The  slight  (though  statistically  nonsignificant)  edge 
held  by  those  who  had  received  aircraft  training 
motivates  one  to  find  ways  in  the  simulator  curriculum 
to  eliminate  or  decrease  that  small  difference.  It 
seems  reasonable  to  shift  emphasis  to  the  question  of 
how  to  maximize  simulator  training;  it  would  seem 
uneconomical  to  use  the  aircraft  in  a  separate  training 
module  just  to  erase  the  small  differences  shown  in 
this  study. 

The  problems  of  simulator  realism  persist  (as  judged  by 
trainee  comments) .  Why  are  sink  rates  higher  in  the 
simulator?  It  is  known  that  these  are  recurrent 
findings  in  simulator  research,  but  it  is  not  known 
why.  This  study,  comparing  the  same  pilots' 
performances  in  simulator  and  aircraft  with  very 
precise  indices,  may  provide  insight  into  transfer 
phenomena.  A  possible  capacity  for  predicting  aircraft 
performance  from  simulator  data  using  a  multiple 
regression  equation  was  indicated. 
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1.  Authors:  Reicher,  Gerald  M. ,  Davidson,  Brian  J., 
Hawkins,  Harold  L.,  &  Osgood,  Gilbert 

2.  Title:  Low  Cost  Simulation  of  Piloting  Tasks 

3.  Source:  Center  for  Cognitive  and  Perceptual  Research, 
University  of  Oregon,  Technical  Report,  January  1980 

4.  Topic  Keywords:  Flight  Simulation  ; 

Instrument  Flight  Simulation  ;  Low  Fidelity  . 

5.  Short  Summary:  An  attempt  to  validate  a  low-cost 
computer-drive  flight  simulator  of  low  physical  fidelity 
produced  some  correlations  of  performance  on  the  device  with 
the  subjects'  training  background.  Results  suggest  the 
usefulness  of  the  simulator  in  measuring  flight  proficiency. 

6.  Device:  Computer  driven  instrument  flight  simulator 
with  3  instruments  resembling  real-aircraft  instruments. 

7.  Institutions: 

a.  Sponsor:  Personnel  and  Training  Research  Programs, 
Psychological  Sciences  Division, 

Office  of  Naval  Research 

b.  Performing  Organization:  Center  for  Cognitive  and 
Perceptual  Research,  University  of  Oregon,  Eugene,  OR 
97403 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  (number  not  specified,  11 
observations  taken  in  the  smal 1 est- sampl e  trial) 
trainees  of  varying  flight  experience  from  Lane 
Community  College  Flight  Technology  Program 

c.  Tests  or  Trials/Timing:  Two  in-process  trials 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  anc  Ratings:  Deviation 
from  prescribec  flight  track,  instrument-recorded  and 
sub-  sequently  judged  by  two  judges  independently 

f.  Experimental  Setting, '’Training  Context:  Laboratory; 
hands-on 

g.  Statistical  Methods:  t-test;  correlation 
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h.  Variables  being  manipulated: 

(1)  Training  Device:  Computer-driven  instrument 
flight  simulator  with  3  instruments  resembling 
real-  aircraft  instruments.  The  training  device 
was  also  the  testing  device. 

(2)  Fidelity  Levels: 

(a)  Physical:  Low 

(b)  Functional:  Medium-high  (the  experiment 
was  actually  an  attempt  to  assess  this 
variable) 

(3)  Type  of  task/skill  required:  cognitive, 
psychomotor,  perceptual,  part-task 

i.  Stage  of  Training:  familiarization,  skill 

j.  Trainee  Sophistication:  novice  and  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  not 
appl i cabl e 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

( 1 '  Intensity:  incomplete 

,2;  Features  used:  Number/Quality  of  Responses; 
others  not  specified 

9 .  Abstract: 

a.  Study  Synopsis:  An  attempt  was  made  to  validate 
the  functional  fidelity  of  an  instrument  flight 
simulator  which  had  low  physical  fidelity,  by 
correlating  performance  on  the  simulator  with  subjects' 
flight  background.  A  computer  program  was  written 
especially  for  the  experiment,  which  subjected  pilots 
to  simulated  wind  and  turbulence.  The  experimental 
task  was  to  follow  a  prescribed  flight  course  in  three 
progressively  more  difficult  wind  and  turbulence 
conditions.  In.  truments  provided  were:  (1)  similar  to 

a  directional  gyro,  (2)  similar  to  a  turn  indicator; 
and  (3)  similar  to  a  course  deviation  indicator  and 
TO, FROM  indicator  of  a  VOR  receiver.  Besides 
performing  the  flight  task  at  three  levels  of 
difficulty  during  initial  trials,  the  pilots  were 
additionally  required  to  perform  the  flight  task  while 
also  performing  auditory  interference,  visual 
interference,  and  cognitive  (cal  .ation)  interference 
tasks . 

The  device  recorded  tne  flight  course  which  was  then 
rated  in  comparison  with  the  prescribed  flight  course 
by  two  judges  independently,  based  "primarily  on 
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accuracy  and  smoothness. " 

The  pilots'  performance  on  the  experiment  task  was  then 
correlated  with  their  flight  training  backgrounds. 

b.  Results:  I 

(1)  Key  Data:  Correlation  between  training  hours 
and  simulator  performance  with  the  simultaneous 
calculation  task  was  -.54  for  the  easier  task  and 
-.68  for  the  difficulty  task.  Since  one  is  the 
highest  rank,  the  negative  correlations  with  hours 
of  training  are  as  expected. 

(2)  Verbal  Description:  A  few  words  of  caution 

are  in  order  about  interpretation  of  these  high 
correlations.  Much  data  were  lost  due  to  computer 
failures  and  subject  attrition  so  that  the  numbers  ) 

of  observations  are  small.  Although  both  of  the 

scores  are  significant  (a  t-test  with  pt.05  was 
used  throughout)  the  actual  values  of  the 
correlations  should  not  be  taken  as  good  estimates 
of  accounted  for  variance. 

•  > 

c.  Authors'  Conclusions:  Tentative  validation  of  the 
simulation  was  obtained.  The  simulator  seems  to  be 
sensitive  to  some  of  the  important  aspects  of  pilot 
training. 

It  seems  likely  that  discovering  good  pilot  ability  is  I 

best  done  with  techniques  using  information  overload. 

Many  pilot  tasks  are  not  demanding  in  most  situations. 

On  instruments,  in  turbulence  with  one  or  two  radios 

out,  and  going  twice  normal  speed  might  make  this  task 

more  challenging  and  discriminating  of  those  who  can 

handle  bad  situations  while  piloting.  This  is  the  sort  I 

of  challenge  that  can  be  set  up  by  a  simulation. 

The  flexible  simulator  also  allows  more  discriminating, 
repeatable,  accurate  measurement. 

» 
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1.  Authors:  Reid,  Gary  B. ,  &  Cyrus,  Michael 

2.  Title:  Transfer  of  Training  with  Formation  Flight 
Trainer 

3.  Source:  AFHRL-TR-74-102 ,  December  1974 

4.  Topic  Keywords:  Flight  Training  ;  Transfer  of  Training 
;  Formation  Flight  Training  ;  Simulation  . 

5.  Short  Summary:  Five  sorties  on  a  high-fidelity  flight 
simulator  were  found  as  effective  as  two  actual-  aircraft 
sorties  in  training  for  formation  flight  in  a  jet  training 
aircraft. 

6.  Devices:  Prototype  high  fidelity 

Formation  Flight  Trainer,  proving  realistic  2-aircraft 
formation  simulation  via  television  projection 

7 .  Insti tutions : 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
AFHRL,  Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  6  (3  for  each  sub-study) 

b.  Description  of  Groups: 

Study  One  - 

(1)  Subjects:  Undergraduate  Pilot  Trainees  with 
approximately  112  hours  jet  aircraft  flight 
training 

(a)  approximately  24  trained  on  Formation 
Flight  Simulator 

(b)  approximately  24  minimally  trained; 
familiarization  with  test  aircraft  only 

(2)  Controls:  approximately  24  same;  trained  by 
normal  syllabus,  on  aircraft 

Study  Two  - 

1.1  Subjects:  Under  g  raauat  e  Pilot  Trainees  with 
approximately  90  hours  total  jet  aircraft  flight 
training,  including  8  hours  formation  flight 

(a. i  16  trained  on  Formation  Flight  Simulator 
(b)  16  minimally  trained;  familiarization 

with  test  aircraft  only 
(2)  Controls:  16  same;  trained  by  normal 
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syllabus,  on  aircraft 

c.  Tests  or  trials/timing:  One  post-test  checkflignt 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  12-point 
performance  scale,  scored  by  Instructor  Pilot  on 
checkflignt 

f.  Experimental  Setting/Training  Context: 
Institutional--  hands-on 

g.  Statistical  Methods:  ANOVA;  "a  posteriori  test" 
(Tukev's  HSD) 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Prototype  high  fidelity 
Formation  Flight  Trainer,  providing  realistic 

2 -aircraft  formation  simulation  via  television 
pro] ection 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified,  presumably 
medium-high  to  high 

(b)  Functional:  high 

(3)  Type  of  task/skill  required:  aircraft 
formation  flight;  operations,  cognitive, 
psychomotor,  motor,  perceptual,  procedural 

(4)  Task  Difficulty:  High 

l.  Stage  of  Training:  familiarization;  skill; 
transition 


Trainee  Sophistication:  intermediate 

Incorporation  of  Device  into  P.O.I.:  lock-step  and 
*  nst  r uctor-managed 

Leer  Acceptance  or  Attitude: 

\I)  Instructors:  not  discussed 
i  2)  Students:  not  discussed 

v. .  L'se  of  Instructional  Features: 

>  1 )  Intensit y  :  ir.tensi’  e 
. Features  used :  net  specified 


■  b  s  t  r  a  c  t : 

a.  Study  Synopsis :  This  study ,  in  two  rcugniy 
identical  ports,  sought  to  evaluate  the  training 
effectiveness  of  a  prototype  high  fidelity  formation 
flight  trainer  simulator  ( FFT )  by  a  transfer  of 
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training  design. 

Subjects  were  trainees  in  the  USAF  Undergraduate  Pilot 
Training  Program  in  transition  from  basic  contact  to 
formation  flight.  Both  parts  of  the  study — "Study  One" 
and  "Study  Two" — were  conducted  identically  except  that 
the  testing  procedures  in  "Study  Two"  were  more 
controlled  and  more  reliable.  Controls  were  students 
trained  via  the  standard  syllabus  which  included  three 
formation-flight  trials  in  aircraft  prior  to 
checkflight  (2  training  trials  and  checkflight 
orientation  run) .  There  were  2  groups  of  experimental 
subjects:  (1)  those  who  received  neither  aircraft  nor 

simulator  training  but  were  given  an  orientation  flight 
in  the  aircraft  prior  to  checkflight;  (2)  those  who 
received  no  aircraft  training  but  5  simulated  formation 
sorties  in  the  FFT  instead,  and  an  orientation  flight 
in  the  aircraft  prior  to  checkflight.  Total,  number  of 
personnel  in  the  study:  118 

Transfer  of  training  was  measured  by  a  12-point 
continuous-scale  grade  on  the  checkflight,  scored  by 
the  Instructor  Pilot. 

b.  Results: 


(1)  Key  Data: 


Differences  Among  Means  of  Performance 


Scores  —  Study  Two  (N=42) 

Groups  Scores 

FFT  Trained  55.31 

Minimally  Trained  43.49 

Syllabus  Trained  53.41 


ANOVA,  Performance 

Scores 

(Study  Two) 

Source 

DF 

MS  F 

Between  Groups 

2 

449.51  5  .33 

Within  Groups 

39 

84.23 

Total 

41 

(p-  .05) 


(2)  Verbal  Description:  Students  trained  in  the 
FFT  scored  significantly  better  than  the 
essentially  untrained  students,  indicating  that 
simulator  training  resulted  in  improved 
performance  in  the  aircraft  (not  just  due  to  the 
pre-checkf light  orientation  flight).  However, 
differences  between  FFT-trained  groups  and  the 
syllabus-trained  groups  fail  to  attain  statistical 
significance. 


c.  Authors'  Conclusions:  Results  from  these  studies, 
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especially  in  light' of  their  repeatability,  provide 
conclusive  evidence  that  the  formation  simulator  is  an 
effective  training  device.  Comparisons  of  checkflight 
scores  lead  to  the  conclusions  that  both  training 
methods,  simulator  and  real-aircraft  formation  flights, 
are  effective  at  this  early  stage  of  the  students' 
skill  acquisition. 

It  is  suggested  that  the  development  of  effective 
instructional  strategies,  the  definition  of  the  amount 
of  useful  simulator  practice,  and  determination  of  how 
the  formation  simulator  can  best  complement  aircraft 
instruction,  will  result  in  a  substantial  increase  in 
the  FFT's  training  effectiveness. 
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1.  Authors:  Riedel,  James  A.,  Abrams,  Macy  L. ,  & 

Post,  David 

2.  Title:  A  Comparison  of  Adaptive  and  Nonadaptive 
Training  Strategies  in  the  Acquisition  of  a  Physically 
Complex  Psychomotor  Skill 

3.  Source:  Navy  Personnel  Research  and  Development  Center 
Training  Report  76-24,  December  1975 

4.  Topic  Keywords:  Welder  Training  ; 

Arc  Welding  Simulation  ;  Reinforcement  Principles  ; 

Adaptive  Training  ;  Training  Effectiveness  . 

5.  Short  Summary:  A  comparison  of  adaptive  and  nonadaptive 
training  strategies  in  teaching  a  complex  psychomotor  skill 
(arc  welding)  finds  no  significant  difference  in  skill 
acquisition  between  the  two  training  conditions. 

6.  Device:  Manual  arc  welding  simulator  (adaption 
controlled  by  experimenter,  not  .automatically) 

7.  Institutions: 

a.  Sponsor:  Navy  Personnel  Research  and  Development 
Center,  San  Diego,  CA  92152 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2  (with  6  subgroups  under  each 
condition) 

b.  Description  of  Groups: 

(1)  Subjects:  30  Hull  Maintenance  Technician 
Fireman  and  Firemen  Apprentice  Trainees  (6 
subgroups,  for  6  levels  of  training  difficulty) 

(2)  Subjects:  30  same,  subgroups  same 

c.  Tests  or  Trials/Timing:  Pre-  and  Post-Test 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  Error 
rates  in  each  of  3  spatial  dimensions 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  Analysis  of  Covariance 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  same  for  both  groups. 
Technique  was  the  variable  of  interest. 

(a)  Adaptive  technique  (machine  tolerances 
tightened  by  experimenter  when  trainee  showed 
improvement) 

(b)  Fixed  technique  (machine  tolerances  not 
varied) 

(2)  Fidelity  Levels: 

(a)  Physical:  Medium 

(b)  Functional:  Low 

(3)  Type  of  Task/Skill  Required:  Arc  welding; 
psychomotor,  part-task 

(4)  Task  Difficulty:  High  (with  6  levels; 
perhaps  Medium-High  to  High) 

i.  Stage  of  Training:  introduction;  skill 

j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable  (experimental) 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  incomplete 

(2)  Features  used:  augmented  feedback 

Abstract: 

a.  Study  Synopsis:  This  study  compared  the  effects  of 
using  an  adaptive  technique  versus  a  fixed  technique  in 
teaching  a  complex  3-dimensional  perceptual  and 
psychomotor  skill  (arc  welding)  on  a  simulator  of  low 
to  medium  fidelity.  A  secondary  purpose  was  to  measure 
the  correlation  between  pre-training  and  post-training 
performance  on  the  apparatus. 

Subjects  were  60  Hull  Maintenance  Technician  Fireman 
and  Fireman  Apprentice  trainees.  Pre-tests  and  post¬ 
tests  on  the  training  apparatus  were  given  to  all 
subjects  at  four  different  levels  of  difficulty. 

Within  each  of  the  two  main  groups  were  six  sub-groups 
trained  at  six  different  levels  of  task  difficulty. 

All  subjects  trained  by  attempting  to  minimize  error  in 
three  spatial  dimensions  simultaneously  (in-out, 
side-to-side,  and  up-  down)  while  moving  a  stylus  over 
a  prescribed  simulated  welding  track.  The  fixed 
technique  subjects  worked  within  fixed  margins  of  error 
throughout  all  100  training  trials;  adaptively  trained 
subjects  were  compelled  to  work  within  progressively 


narrower  margins  as  their  performance  improved. 

Measures  taken  in  the  pre-tests  and  post-tests  were  of 
error  in  the  three  spatial  dimensions. 

b.  Results:  Results  of  analysis  for  three  modes 
(stylus  tip-to-target  distance,  tracking,  and  these  two 
variables  combined)  yield  no  significant  differences 
among  various  levels  of  practice  difficulty.  The 
correlation  between  variate  and  covariate  (pretrials) 
for  stylus  tip-to-target  surface  distance  was  .71,*  for 
tracking,  .69;  for  the  combined  variables,  .74. 

The  effects  of  training  schedules  (adaptive  and 
nonadaptive)  were  not  found  to  be  significantly 
different,  either.  All  reliable  measured  dimensions 
showed  no  significant  differences  in  test  performance 
between  the  various  levels  of  practice  difficulty. 
Similarly,  the  effects  of  the  training  schedules, 
adaptive  and  nonadaptive,  were  not  found  to  be 
significantly  different.  While  neither  of  the  main 
effects  were  significant,  the  average  percent  of 
improvement  between  pre-  and  post-tests  for  all 
W)  •  subjects  was  65.58%  out  of  a  possible  100%.  Results 

also  suggest  no  significant  linear  trend  for  adaptive 
and  nonadaptive  conditions  across  levels  of  practice 
difficulty . 

c.  Authors'  Conclusions:  Two  primary  hypotheses  can 
be  advanced  to  explain  why  the  adaptive  training 
condition  did  not  prove  superior  to  the  nonadaptive: 

(1)  the  subjects  in  the  adaptive  mode  were  not  able  to 
detect  progress  because  their  error  signal  rates  were 
held  constant,  and  thus  were  subjected  to  negative 
motivational  effects,  and  (2)  adaptive  techniques  are 
not  applicable  in  the  present  case  due  to  the 
complexity  of  the  task.  The  second  hypothesis  seems 
more  plausible  in  explaining  the  present  results. 

Abrams  et.  al.  (1974)  found  that  physically  complex 
psychomotor  skills  are  different  than  simple 
psychomotor  skills  and  training  strategies  or 
techniques  should  consider  the  task  complexity.  Even 
tasks  which  appear  to  be  simple  can  be  quite  complex 
because  the  psychological  processes  involved  can  range 
to  higher  forms  of  information  processing.  In  the 
present  study,  the  task  may  have  involved  processes 
that  do  not  occur  in  many  less  physically  complex 
tasks . 

In  the  present  case,  a  training  technique  that  has  been 
successfully  employed  with  other  psychomotor  skills  did 
not  generalize  to  a  more  physically  complex  task,  thus 
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indicating  a  limitation  of  that  strategy.  Further 
research  is  recommended  on  the  interaction  between 
physical  task  complexity  and  utility  of  adaptive 
training  strategy. 

For  teaching  the  subject  task,  a  fixed  schedule,  being 
cheaper,  is  recommended  as  the  most  cost-efficient 
method. 

Simulator  pretraining  performance  measures  were  highly 
correlated  with  simulator  final  performance.  (R  = 

.74).  Since  final  simulator  performance  has  previously 
been  found  to  correlate  with  welding  performance  (R  = 
.79) ,  this  suggests  a  potential  use  of  the  apparatus  as 
a  predictive  instrument  for  arc  welding. 
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1.  Authors:  Rigney,  J.  W. ,  Towne,  D.  M. ,  Moran,  P.  J. ,  & 
Mishler,  R.  A. 

2.  Title:  Field  Evaluation  of  the  Generalized  Maintenance 

Trainer-Simulator:  II.  AN/SPA-66  Radar  Repeater 

3.  Source:  Behavioral  Technology  Laboratories  Technical 
Report  No.  90,  Department  of  Psychology,  University  of 
Southern  CA,  November  1978 

4.  Topic  Keywords:  Maintenance  Training  Simulation  ; 
Electronics  Troubleshooting  . 

5.  Short  Summary:  The  versatility  of  a  Generalized 
Maintenance  Training  Simulator  to  provide  instruction  on 
different  types  of  electronics  equipment  is  validated  in  a 
transf er-of-training  study  in  the  field. 

6.  Devices: 

a.  Simulator: 

Generalized  Maintenance  Training  Simulator  ,  a 
computer-based  simulator  that  automatically  selects 
malfunctions  and  displays  high-resolution  color  images 
of  the  equipment.  Student  interacts  by  touching  with  a 
small  stylus. 

b.  Actual  Equipment:  AN/SPA  Radar  Repeater 

7.  Institutions: 

a.  Sponsor:  Naval  Personnel  Research  and  Development 
Center,  and  Personnel  and  Training  Research  Programs, 
Psychological  Sciences  Division,  Office  of  Naval 
Research,  and  Advanced  Research  Projects  Agency 

b.  Performing  Organization:  Behavioral  Technology 
Laboratories,  Department  of  Psychology,  University  of 
Southern  California 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  Subjects:  10  Navy 
electronics  technicians 

c.  Tests  or  Trials/Timing:  In-process  trials  of  33 
problems;  post-test  of  4  problems 

d.  Number  of  Different  Types  of  Measures:  1 


e.  Description  of  Measurements  and  Ratings:  Time  to 
complete  problems 

f.  Experimental  Setting/Training  Context:  laboratory, 
hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Generalized  Maintenance 
Training  Simulator 

(2)  Fidelity  Levels: 

(a)  Physical:  medium-low  (2-dimensional 
microfiche  projections  of  equipment) 

(b)  Functional:  Medium-high 

(3)  Type  of  Task/Skill  Required:  Maintenance 
troubleshooting;  cognitive;  perceptual 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  familiarization,  skill 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  specified  (experimental 
personnel) 

(2)  Students:  very  favorable  as  indicated  by 
questionnaire 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified,  presumed  intensive 

(2)  Features  used:  Restart/Resequence  Capability; 
Malfunction  Selection;  Sign-in  Capability; 
Number/Quality  of  Responses;  Next  Activity 
Feature;  Automated  Demonstration 

Abstract : 

a.  Study  Synopsis:  This  study  was  intended  primarily 
to  test  the  validity  of  two  premises  fundamental  to  the 
development  of  a  Generalized  Maintenance  Training 
Simulator  (GMTS) ,  namely: 

(1)  that  the  system  is  general,  and  can  therefore 
deliver  training  and  simulation  on  any  target 
system  properly  documented; 

(2)  that  the  data  base  can  be  prepared  by  technial 
personnel  who  are  experts  only  in  the  behavior  of 
the  target  system  and  the  format  of  the  required 
data  base,  but  are  not  concerned  with  the  nature 
of  th  GMTS  program. 


This  study  was  not,  therefore,  designed  to  evaluate  the 
training  effectiveness  of  the  device,  and  only  loose 
control  was  exercised  in  the  selection  of  subjects  and 
problems. 

The  device  itself  is  an  interactive  computerized  system 
consisting  of  hardware  and  software  as  follows:  (1) 

CRT  which  displays  dynamic  information  about  a 
simulated  problem;  (2)  microfiche  viewer,  which 
displays  color  images  of  the  target  system;  (3)  touch 
input  device  (stylus)  which  provides  student  entry 
capability  via  touching  the  desired  loction  on  the 
projected  images  of  simulated  equipment,  or  touching 
the  commands  on  the  command  menu;  (4)  10K 
minicomputer;  (5)  AED  floppy  disk  drive  mass  data 
storage;  (6)  Cybernex  (custom  design)  interface;  and 
(7)  CJCSD  PASCAL  language. 

The  instructional  format  of  the  system  remains 
invariable,  whereas  the  content  varies  according  to  the 
equipment  being  simulated.  Previously,  the  GMTS  had 
been  used  experimentally  to  simulate  a  large, 
multi-equipment  maintenance  system  for  radio 
communication.  The  present  study  involved  the 
AN/SPA-66  radar  repeater,  an  indicator  group  which 
operates  in  conjunction  with  other  radar  equipment  to 
provide  a  central  display  of  radar  returns.  AN/SPA-66 
equipment  data  was  prepared  entirely  by  two  technical 
experts  who  were  concerned  only  with  supplying  the 
specified  data  on  the  required  format,  who  were  not 
concerned  with  the  nature  of  the  GMTS  program. 

Ten  Navy  electronics  technicians  were  trained  in 
troubleshooting  on  the  device,  and  subsequently  tested 
on  actual  equipment.  Their  times  to  solution  on  both 
training  trails  and  transfer  test  problems  were 
recorded. 

Student  attitude  was  measured  by  questionnaire. 


b.  Results: 


(1)  Key  Data: 

Problem  Time-to-solution  SD 


Practice  1 
Practice  2 
1 
2 

3  ' 

4 

5 


(in  minutes) 
2.0 
1.3 
1.2 
0.8 
0  .77 
0  .6 
0  .7 


(in  minutes) 
1.8 
1.1 
1.3 
0  .6 
0  .4 
0.3 
0.25 


Note:  Figures  are  reviewers'  extrapolations  from 

a  graph. 

(2)  Verbal  Description:  None  of  the  students 
encounted  discernible  difficulties  in 
transitioning  from  GMTS  to  the  test  on  the  actual 
equipment.  The  figures  above  show  the  mean  times 
and  standard  deviations  for  time-to-completion  of 
practice  problems  and  related  test  problems. 

The  student  attitude  questionnaire  obtained 
generally  very  favorable  responses  only  .04  of 
responses  were  "neutral",  and  none  were 
unfavorable.  More  than  half  of  the  16  respondents 
rated  the  GMTS  either  "Very  Favorable"  or 
"Favorable." 


c.  Authors'  Conclusions:  Since  the  sample  size  is 
extremely  small,  and  problem  difficulty  was  not  a 
controllable  variable,  these  results  are  only  general 
indicators  of  training  effectiveness. 

The  study  demonstrated  that  equipment  experts  can 
effectively  produce  the  data  base  required  by  GMTS  to 
drive  a  comprehensive  training  simulation,  and  the 
generality  of  the  training  system  was  validated  by  use 
of  a  target  system  dissimilar  to  that  previously 
simulated  in  an  earlier  experiment. 

It  appears  from  the  limited  data  that  troubleshooting 
skills  derived  from  GMTS  training  can  be  transferred  to 
the  actual  equipment. 
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1.  Authors:  Roscoe,  Stanley  N.  ,  &  Williges,  Robert  C. 

2.  Title:  Motion  Relationships  in  Aircraft  Attitude  and 
Guidance  Displays:  A  Flight  Experiment 

3.  Source:  Human  Factors  ,  1975,  17(4),  374-387 

4.  Topic  Keywords:  Flight  Simulation  ;  Aircraft  Attitude 
;  Vestibular  Cues  ;  Flight-Director  Displays  ; 

Guidance  Displays  ;  Motion  Cues  . 

5.  Short  Summary:  A  experiment  was  conducted  using  various 
flight-  director  displays  in  a  laboratory  flight  simulator, 
with  sometimes  conflicting  visual  and  vestibular  cues  given 
to  non-pilot  subjects.  Indications  are  that  results  from 
experiments  conducted  either  in  fixed-base  simulators  or  in 
ones  providing  highly  distorted  motion  cues  cannot  be 
trusted  if  spatial  orientation  is  a  central  consideration. 

6.  Devices:  Beechcraft  C-454  Flying  Laboratory 

7.  Institutions: 

a.  Sponsor:  Office  of  Naval  Research 

b.  Performing  Organization:  Aviation  Research 
Laboratory  of  the  University  of  Illinois  at 
Urbana- Champaign 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  16  non-pilot, 
non-pilot-candidate  Naval  ROTC  students 

c.  Tests/Trials/Checkf lights  and  Timing  thereof: 
In-process  Trials 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  Tracking 
Error;  recorded  manually  when  automatic  recording 
equipment  was  found  deficient 

f.  Experimental  Setting/Training  Context: 

Institutional  Laboratory,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Beechcraft  C-454  Flying 
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Laboratory 

(2)  Fidelity  Levels: 

(a)  Physical:  unspecified 

(b)  Functional:  unspecified;  presumably 
high 

(3)  Type  of  Task/Skill  Required:  aircraft 
piloting;  cognitive;  psychomotor 

(4)  Task  Difficulty:  Medium-high 

(5)  Skills  required  by  task:  psychomotor; 
perceptual;  cognitive 

i.  Stage  of  Training:  introduction  (experimental); 
skill 

j.  Trainee  Sophistication:  novice  (very) 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable;  continuous  test 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  applicable 

(2)  Students:  not  applicable 

m.  Use  of  Instructional  Features:  intensive  (test, 
not  instruction) 

Abstract : 

a.  Study  Synopsis:  This  study  investigated  the  impact 
of  several  combinations  of  visual  and  vestibular  cues 
on  flight  maneuver  performance  in  a  laboratory  flight 
simulator . 

The  Beechcraft  C-454  flying  laboratory  can  provide 
several  different  flight-director  display 
configurations  in  either  a  pursuit  mode  or  compensatory 
mode.  An  experiment  was  conducted  with  16  non-pilot 
(non-pilot-candidate)  Naval  ROTC  students  to 
investigate  the  effects  of  using  these  different 
configurations,  often  in  the  presence  of  conflicting 
kinesthetic  and  vestibular  cues,  and  thereby  to 
determine  an  order  of  merit  for  the  different  displays. 

A  series  of  complex  flight  tasks  were  given  to  subjects 
in  two  basic  modes--compensatory  and  pursuit--ar.d  in 
four  different  flight  director  display 
conf igur ations--the  conventional  moving  horizon 
(inside-out),  moving  airplane  ( outside- in) ,  a  hybrid 
frequency-separated  presentation  applying  the  aileron 
position  signal  to  the  aircraft  symbol  to  provide  an 
immediate  indication  of  imminent  bank  angle  changes, 
and  kinalog  (time-lagged  frequency  separtion) .  The 
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checkflight  criterion  scores 


f.  Experimental  Setting/Training  Context:  School 

g.  Statistical  Methods:  Unspecified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  2F87F  High-fidelity  Flight 
Simulator 

(2)  Fidelity  Levels: 

With-moticn  No-motion 
Physical  high  high 

Functional  high  intermediate-high 

(3)  Type  of  Task/Skill  Required:  aircraft  flight; 
cognitive,  psychomotor,  procedural 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  familiarization;  skill 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.;  unspecified; 
probably  lockstep 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  Good  for  with-motion;  not  so 
good  without  motion 

(2)  Students:  Same  as  instructors 

m.  Use  of  Instructional  Features:  Intensive 
Abstract : 

a.  Study  Synopsis:  This  study  was  prompted  by 
instructor  pilots'  subjective  observations  of 
deficiencies  in  the  Training  Device--that  it  did  not 
contribute  to  learning  the  final  phase  of  landing;  and 
that  in  the  without-  platform-motion  mode,  it  failed  to 
give  adequate  fidelity  and  may  have  caused  motion 
sickness.  The  study  was  designed  to: 

(1)  Determine  whether  students  who  did  not  receive 
landing  training  in  the  simulator  would  perform 
just  as  well  in  checkflight  landings  as  those  who 
did  (which  would  confirm  the  irrelevance  of  the 
simulated  landing  training  to  the  aircraft  flight 
landing  task ) ; 

(2)  Determine  whether  lack  of  simulated  cockpit 
motion  would  produce  a  decrement  in  training; 

(3)  Determine  whether  motion  sickness  would  result 
from  an  absence  or  presence  of  simulated  cockpit 


1.  Authors:  Ryan,  Leonard  E.,  Scott,  Paul  G. ,  & 

Browning,  Robert  F. 

2.  Title:  The  Effects  of  Simulator  Landing  Practice  and 
the  Contribution  of  Motion  Simulation  to  P-3  Pilot  Training 

3.  Source:  DTIC  TAEG  Report  No.  63,  September  1978 

4.  Topic  Keywords:  Landing  Training  ;  Motion  Simulation  ; 
Transfer  of  Training  ;  Flight  Simulation  ; 

Training  Effectiveness  . 

5.  Short  Summary:  A  study  evaluating  the  2F87F  Flight 
Simulator  showed  it  effective  in  training  the  final 
touchdown  portion  of  the  landing  task.  There  were  also  some 
indications  that  simulated  cockpit  motion  did  not  contribute 
in  any  large  practical  way  to  learning  most  simulated  flight 
maneuvers,  and  that  little  evidence  of  motion  sickness 
resulted  from  simulator  experience  either  with  or  without 
cockpit  motion. 

6.  Devices:  2F87F  High-Fidelity  Flight  Simulator 

7.  Institutions: 

a.  Sponsor:  Training  Analysis  and  Evaluation  Group, 
Orlando,  FL  32813 

b.  Performing  Organization:  Same 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups: 

(1)  Study  One  (landing  training):  4 

(2)  Study  Two  (motion  vs.  no-motion) :  2 

b.  Description  of  Groups: 

(1)  Study  One: 

(a)  Controls:  three  groups  of  first-tour 
Naval  aviators,  who  had  completed 
undergraduate  training,  numbering  27,  39,  and 
10  respectively 

(b)  Experimental  subjects:  19  same 

(2)  Study  Two: 

(a)  Controls:  one  group  of  39,  same  as  Study 
One 

(b)  Experimental  Subjects:  one  group  of  11  , 
same 

c.  Tests  or  Trials/Timing:  Post-test  checkflights 

d.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  Post-test 
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is  evidence  from  this  study  that,  at  least  for 
experienced  pilots,  the  absence  of  realistic  motion 
acceleration  induces  a  relatively  lasting  decrement  in 
performance.  It  appears  that  kinetic  cueing  should  be 
provided  when  training  pilots  to  react  to  emergencies 
even  if  the  nature  of  the  emergency  is  initially 
unrelated  to  the  dynamics  of  the  vehicle,  since  no 
consistent  reliable  relationships  between  the  cues  and 
the  emergency  were  reflected  in  pilot  response.  This 
supports  the  hypothesis  that  the  kinetic  cues  serve 
primarily  an  "alerting"  function. 

Superiority  of  the  kinetic  training  condition 
manifested  itself  in  the  deterioration  of  the 
performance  of  the  statically  trained  pilots;  there 
was  no  appreciable  evidence  of  differences  in  learning 
for  either  of  the  main  experimental  groups  during  the 
"training"  cycle. 
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b.  Results: 


(1)  Key  Data: 

(Out  of  30  Plights) 

Proportions  of  Successful  Criterion 
Touchdowns  as  a  Function  of  Number 
of  Training  Trials 

"KINETIC"  GROUP  "STATIC"  GROUP 

Training  Training 


Trials 

Successes 

% 

Trials 

Successes 

% 

z 

P 

15 

18 

60 

15 

13 

43 

1.42 

_ 

30 

13 

43 

30 

8 

27 

1.33 

- 

45 

19 

63 

45 

10 

33 

2.75 

<  .01 

60 

18 

60 

60 

19 

63 

.25 

- 

75 

18 

60 

75 

9 

30 

2.50 

<.05 

Total 

86 

57 

Total 

59 

39 

2.14 

<.05 

(2)  Verbal  Description:  In  respect  to  approach 
performance,  there  were  significant  differences 
between  the  kinetically  and  statically  trained 
groups  on  5  of  the  10  measures  during  the 
criterion  flights.  In  every  case,  performance  of 
the  kinetically  trained  group  was  significantly 
better  than  that  of  the  statically  trained  group. 
During  training,  as  the  number  of  training  trials 
increased,  performance  deteriorated  for  the 
statically  trained  group,  on  all  measures  except 
mean  stick  displacement  where  no  difference 
occurred. 

In  respect  to  the  measures  taken  at  touchdown,  the 
table  above  shows  the  differences  between  the 
groups  on  the  criterion  flights,  as  a  function  of 
the  number  of  training  trials.  The  kinetically 
trained  group  outperformed  the  statically  trained 
group  significantly  overall,  and  on  two  of  the 
five  levels  of  training. 

The  superiority  of  kinetic  training  did  not  seem 
generally  to  be  significantly  related  to  the  kind 
of  emergency  regime  simulated;  however,  two  of 
the  failure  regimes  did  show  a  significant  effect 
on  two  performance  measures,  and  one  of  the 
regimes  was  correctly  recognized  by  pilots 
significantly  more  often  under  kinetic  conditions. 

c.  Authors'  Conclusions:  Kinetic  cueing  is  a  valuable 

adjunct  to  airborne  vehicle  simulation  systems.  There 
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Abstract : 
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a.  Study  Synopsis:  This  study  sought  to  evaluate  the 
contribution  of  motion  cues  in  training  experienced 
pilots  to  land,  in  simulation,  a  high-performance  jet 
aircraft  on  a  carrier  deck.  Its  implications  for 
training  in  general  are  limited  by  the  following 
considerations:  (1)  all  subjects  were  experienced 

pilots,  having  a  median  2,803  hours  flying  experience, 
and  all  had  previously  flown  carrier  landings;  (2) 
since  all  subjects  received  at  least  10  trial  flights 
with  motion  in  the  simulator  before  "training"  began, 
what  was  under  study  was  not  acquisition  of  skill,  but 
rather  what  decrement  of  performance  would  result  from 
"training"  in  the  no-motion  condition  after  exposure  to 
motion;  (3)  the  "criterion"  trials  testing  was  done  in 
the  simulator;  no  "transfer"  test  to  aircraft  was 
conducted;  (4)  the  simulation  primarily  involved 
"emergencies"  of  various  kinds  rather  than  "normal" 
flight. 

The  12  experienced  pilot  subjects  were  divided  into  6 
groups  of  2  each.  The  "control"  pair  flew  all  trials 
with  motion.  The  5  experimental  pairs  flew  10  initial 
trials  with  motion,  and  then  each  pair  flew  a  different 
number  of  "training"  trials  without  motion.  All  groups 
were  then  tested  in  "criterion"  trials  with  motion. 
There  were  no  significant  differences  between  any  of 
the  pairs  on  3  performance  measures  taken  during  the 
initial  10  trials.  Measures  taken  included  pilot  and 
system  performance  measures  during  approach  and 
touchdown;  and  a  post-flight  attempt  by  the  pilot  at 
identification  of  which  simulated  emergency  failure 
regime  he  had  been  landing  under  (there  were  nine 
possibilities) . 


I 


I 
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e.  Description  of  Measurements  and  Ratings: 

(1)  Pilot  performance  measures 

(2)  System  performance  measures 

(3)  Terminal  flight  (touchdown)  measures 

(4)  Recognition  of  failure  regime 

f.  Experimental  Setting/Training  Context:  laboratory, 
hands-on 

g.  Statistical  Methods:  t-tests;  ANOVA 

h.  Variables  Being  Manipulated: 

(1)  Training  Device:  High  fidelity  visual  flight 
simulator  of  high  performance,  carrier-based  jet 
aircraft,  capable  of  3  degrees  of  freedom  platform 
motion. 

(a)  With  motion  -  "kinesthetic” 

(b)  Without  motion  -  "static" 

Chief  variable  was  number  of  training  trials  flown 
without  motion. 

(2)  Fidelity  Levels: 

With  Motion  Without  Motion 

Physical  Medium  high  Medium 

Functional  Medium  high  Medium 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  skill  (experimental  only) 

j.  Trainee  Sophistication:  intermediate  to  expert 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  not  applicable 

(2)  Students:  favorable  as  measured  by 
questionnaire 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  Number/quality  of  response; 
others  not  specified 
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1.  Authors:  Ruocco,  Joseph  N. ,  Vitale,  Patrick  A.,  & 
Benfari,  Robert  C. 

2.  Title:  Kinetic  Cueing  in  Simulated  Carrier  Approaches 

3.  Source:  NAVTRADEVCEN  Technical  Report  1432-1,  28  April 
1965 


4.  Topic  Keywords:  Kinetic  Cueing  ;  Flight  Simulation  ; 
Carrier  Landing  Simulation  ;  Platform  Motion  . 

5.  Short  Summary:  A  study  of  the  contribution  of 
kinesthetic  cueing  via  platform  motion  in  simulated  training 
of  experienced  pilots,  finds  that  in  the  carrier  landing 
task,  the  static  (no  motion)  condition  degrades  performance 
whereas  the  motion  condition  sustains  performance  on 
criterion  trials  tests  done  in  the  simulator. 

6.  Devices:  High  fidelity  visual  flight  simulator  of  high 
performance,  carrier-based  jet  aircraft,  capable  of  3 
degrees  of  freedom  platform  motion 

7.  Institutions: 

a.  Sponsor:  U.S.  Naval  Training  Device  Center,  Port 
Washington,  NY  11050 

b.  Performing  Organization: 

Grumman  Aircraft  Engineering  Corporation,  Bethpage, 
Long  Island,  NY 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  6 

b.  Description  of  Groups:  All  subjects  and  controls 
were  pilots  with  median  number  of  2,803  hours  flight 
experience,  and  some  experience  at  carrier  landings; 
they  were  matched  on  the  basis  of  three  performance 
measures  on  initial  trial  "flights"  in  the  simulator 
with  no  significant  differences  between  scores 

(1)  Subjects:  2,  75  training  trials;  no  motion 

(2)  Subjects:  2,  60  training  trials  "  " 

(3)  Subjects:  2,  45  training  trials  "  " 

(4)  Subjects:  2,  30  training  trials  "  " 

(5)  Subjects:  2,  15  training  trials  "  " 

(6)  Controls:  2,  75  training  trials  with  motion 


c.  Tests  or  Trials/Timing:  in-process  trials  and 
post- tests 

d.  Number  of  Different  Types  of  Measures:  4 
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c.  Authors'  Conclusions:  The  groups  trained  to  the 
70%  proficiency  level  displayed  positive  transfer 
during  later  stages  of  the  transfer  exercise.  The 
important  determinant  in  transfer  appeared  to  be  the 
amount  of  training  received  rather  than  the  nature  of 
the  device  on  which  practice  was  given.  It  appears 
that  the  amount  of  training  has  both  a  positive  and 
negative  impact  on  transfer.  It  is  positive  in  the 
sense  that  more  highly-trained  students  can  make  better 
use  of  feedback  and  can  eventually  reach  a  higher 
performance  level,  and  negative  in  that  they  may  have 
learned  the  "wrong"  lead  and/or  method  of  applying  lead 
for  the  task  at  hand,  and  consequently,  do  poorly  at 
first. 

Clear  specifications  of  the  objectives  of  training  are 
required.  If,  for  instance,  the  objective  is  merely 
familiarization,  setting  criterion  levels,  rather  than 
providing  simply  a  fixed  number  of  trials,  would  be 
wasteful . 
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Results  : 


(1)  Key  Data:  Figure  5  on  p.  13  of  the  report 
contains  the  most  important  information.  Data 
presented  in  table  below  were  approximate  readings 
from  the  graphs: 

Percent  Hits  During  Transfer  Test  as  a 
Function  of  Trial  Block  and  Device 


Machine-Gun  Trained  Group 


of iciency 

Block 

Block 

Block 

Block 

Level 

1 

2 

3 

4 

30% 

47 

39 

61 

57 

50% 

45 

50 

57 

72 

70% 

43 

60 

60 

73 

Control 

48 

55 

60 

54 

Laser  Trained 

Group 

30% 

55 

48 

58 

65 

50% 

50 

49 

67 

53 

70% 

46 

54 

54 

73 

Control 

48 

55 

60 

54 

(2)  Verbal 'Description : 

Training  trials  to  criterion  showed  no  significant 
differences  due  to  device.  The  trials-to- 
criterion  for  group  trained  to  70%  proficiency 
were  significantly  higher  than  either  the  30%  or 
the  50%  group. 

On  the  transfer  test,  significant  differences  in 
skill  due  to  device  were  not  demonstrated.  All 
groups  improved  performance  during  the  course  of 
the  4-block  transfer  test,  showing  the  test  as 
itself  a  training  session.  The  accuracy  data 
showed  that  in  the  initial  transfer  test  blocks, 
no  trained  group  was  significantly  more  accurate 
than  the  untrained  control  group  (see  table 
above) .  The  groups  trained  to  70%  proficiency, 
however,  regardless  of  training  device, 
demonstrated  significantly  greater  accuracy  at  the 
concluding  block  of  the  transfer  test.  Analysis 
of  "miss"  data  suggested  different  learning 
strategies  were  being  used  by  the  different 
groups,  during  the  course  of  the  transfer  test. 
There  was  some  evidence  of  negative  transfer  from 
the  training  devices. 


k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 
,  (2)  Features  used:  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  The  purpose  of  this  experiment  was 
.to  evaluate  the  training  effectiveness  of  two  tank 
gunnery  training  systems.  An  additional  goal  was  to 
determine  the  level  of  proficiency  which  students 
should  reach  on  the  device  in  order  to  maximize  initial 
transfer  on  the  main  gun.  The  devices  used  to  simulate 
tracking  and  firing  of  the  tank  gun  were  both  mounted 
on  the  tank:  one  was  a  low-power  gas  laser  which 
represented  a  shell  burst  by  a  red  light  return  from 
the  target;  this  device  also  simulated  flight  time  of 
a  real  shell  fired  at  a  moving  target.  The  second 
device,  a  coaxial  machine  gun,  gave  higher  fidelity  to 
a  real  tank  gun  than  the  laser,  in  requiring  greater 
w*  safety  precautions,  making  a  noise  when  fired,  and 

making  a  hole  in  the  target. 

Six  experimental  groups  of  22  tank  gunnery  trainees 
each  were  trained  by  firing  the  two  devices.  Three 
groups  were  trained  on  the  laser  device  to  proficiency 
levels  of  30,  50  and  70  per  cent,  separately;  the 
other  three  groups  were  trained  on  the  coaxial  machine 
gun  to  the  same  respective  levels  of  proficiency.  The 
numbers  of  trials  to  criterion  for  all  groups  were 
recorded.  Following  the  training  on  the  simulators, 
the  groups  were  tested  for  transfer  by  firing  an  actual 
tank  gun.  A  seventh,  control  group,  in  the  same  phase 
of  training  as  the  subjects,  fired  the  acutal  tank  gun 
without  prior  training  on  any  device.  Three 
measurements  were  taken  of  the  live-firing  transfer 
test:  (1)  percentage  of  hits;  (2)  evaluation  of 

misses,  by  type;  and  (3)  speed  of  fire. 

Extensive  matching  of  subjects,  and  control  and  counter 
balancing  procedures  were  undertaken  to  minimize 
confounds. 


level 

(3)  Subjects:  22  same,  trained  to  70%  proficiency 
level 

(4)  Subjects:  22  same  but  trained  on  coaxial 
machine  gun,  trained  to  30%  proficiency  level 

(5)  Subjects:  22  same  as  (4)  but  trained  to  50% 
proficiency  level 

(6)  Subjects:  22  same  as  (4)  but  trained  to  70% 
proficiency  level 

(7)  Controls:  22  same,  given  no  simulated  firing 
training 

c.  Tests  or  Trials/Timing:  1  post-test  (tank  gun 
firing) 

d.  Number  of  Different  Types  of  Measures:  4 

e.  Description  of  Measurements  and  Ratings: 

(1)  Number  of  training  trials  to  criterion 

(2)  Pet.  hits  on  live-fire  tank  transfer  test 

(3)  Evaluation  of  misses  on  live-fire  tank 
transfer  test 

(4)  Speed  of  firing  on  live-fire  tank  transfer 
test 

f.  Experimental  Setting/Training  Context: 

Institutional,  hands-on 

g.  Statistical  Methods:  T-test;  ANOVA;  Arcsin 
square  root  transformation 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  M60A1/3A102B  Laser--see  section  6  above 

(b)  M60A1/M73  Coaxial  Machine  Gun — see 
section  6  above 

(c)  M60A1  Tank  Gun  (transfer  testing  device) 

(2)  Fidelity  Levels  of  Simulators: 

Laser  Coax  Machine  Gun 

Physical  Medium-low  Medium 

Functional  Medium  Medium-high 

(3)  Type  of  Task/Skill  Required:  tank  gunnery; 
cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  medium 

i.  Stage  of  Training:  familiarization,  skill, 
transition 

j.  Trainee  Sophistication:  novice 
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1.  Authors:  Rose,  A.  M. ,  Wheaton,  G.  R. , 

Leonard,  Russell  L.,  jr.,  Fingerman,  P.  W. ,  &  Boycan,  G.  G. 

2.  Title:  Evaluation  of  Two  Tank  Gunnery  Trainers 

3.  Source:  U.S.  Army  Research  Institute  Memorandum  76-19, 
August  1976 

4.  Topic  Keywords:  Transfer  of  Training  ;  Interference  ; 
Negative  Transfer  ;  Tank  Gunnery  Training  ;  Fidelity  ; 
Analytical  Models  ;  Medium-low  Fidelity  . 

5.  Short  Summary:  A  study  of  training  of  tank  gunnery 
trainees  on  devices  differing  in  degree  of  fidelity, 
followed  by  transfer  trials  on  real  tank  guns,  yielded  no 
significant  differences  between  devices  in  training 
effectiveness,  but  presents  examples  of  both  positive  and 
negative  transfer  from  simulated  training. 

6.  Devices: 

a.  M60A1/3A1 02B  Laser  (trainer) :  a  low-power  gas 
laser  mounted  in  an  M73  machine  gun  bracket  in  the 
M60A1  tank.  Red-light  return  represents  shell  burst; 
device  simulates  flight  time  of  real  shell  fired  at 
moving  target. 

b.  M60A1/M73  Coaxial  Machine  Gun,  single  shot  mode 

(trainer):  operational  machine  gun  with  interrupter 

mechanism  to  restrict  fire,  mounted  in  the  M60A1  tank. 
Higher  fidelity  than  device  a. 

c.  M60A1  Tank  Gun  (transfer  testing  device — actual 
equipment) 

7.  Institutions: 

a.  Sponsor:  Army  Research  Institute 

b.  Performing  Organization:  Army  Research  Institute; 
American  Institutes  for  Research 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  7 

b.  Description  of  Groups: 

(1)  Subjects:  22  11E10  tank  trainees  from  D 
Company,  3rd  Battalion,  1st  Training  Brigade  at 
Fort  Knox;  trained  on  laser,  to  30%  proficiency 
level 

(2)  Subjects:  22  same,  trained  to  50%  proficiency 
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(2)  Verbal  Description:  As  above;  plus:  the 
single  reliable  effect  revealed  by  the  command 
flight  path  tracking  performances  was  the 
consistent  superiority  of  pursuit  over 
compensatory  tracking. 

c.  Authors'  Conclusions:  The  moving  airplane  attitude 
presentation  yielded  reliably  worse  disturbed  attitude 
tracking  than  either  the  conventional  moving  horizon  or 
its  frequency-separated  counterpart.  This  single, 
unprecedented  finding  cast  doubt  upon  the  validity  of 
the  results  of  numerous  experiments  in  fixed-base  and 
moving-base  simulators  that  have  indicated  superior 
performance  with  moving  airplane  attitude  displays. 

The  frequency-separated  display  provides  at  least 
equivalent  pilot  steering  performance  to  that  obtained 
with  the  conventional  moving  horizon  format. 

It  is  evident  that  the  results  of  experiments  conducted 
either  in  fixed-base  simulators  or  in  ones  providing 
highly  distorted  motion  cues  cannot  be  trusted  if 
spatial  orientation  is  a  central  consideration. 

Finally,  because  a  within-subj ect  experimental  design 
was  employed,  and  because  an  internal  analysis  of  the 
data  obtained  suggests  the  likelihood  of  differential 
intraserial  transfer  among  displays  on  the  initially 
presented  disturbed  attitude  tracking  task,  the  results 
for  this  task  cannot  be  presented  without  reservation. 

A  direct  comparison  of  the...  (various)  presentations 
employing  independent  groups  of  flight-naive  subjects 
is  essential,  as  is  a  flight  investigation  of  the 
transition  of  experienced  pilots  to  the 
frequency-separated  display. 


sequence  of  tasks  was  counterbalanced. 

One  of  the  most  important  tasks  was  a  disturbed 
attitude  tracking  task,  designed  explicity  to  reveal 
differences  in  performances  inherently  associated  with 
attitude  presentation.  The  subject  was  forced  to 
respond  directly  to  visually  presented  attitude 
indications  that  were  in  conflict  with  supraliminal 
vestibular  cues  of  angular  acceleration. 

b.  Results: 

(1)  Key  Data: 

Mean  Standard  Deviation  of  disturbed  attitude 
tracking  error  in  arbitrary  units  for  two  trials 
by  sub-groups  of  four  subjects  each  on  each  of 
four  display  types  across  serial  positions: 


DISPLAY 

TYPE 

Moving 

Moving 

Frequency 

Trial 

Horizon 

Airplane 

Separated 

Kinalog 

Serial 

Position 

One 

1 

2.34 

2.26 

1 .76 

2.87 

2 

2 .03 

1.77 

1.27 

1.59 

Serial 

Position 

Two 

1 

1 .07 

2  .08 

1  .20 

1  .92 

2 

1 .12 

1.81 

1.01 

1  .61 

Serial 

Position 

Three 

1 

1.15 

1.34 

1 .92 

1  .91 

2 

1 .22 

1.77 

1.33 

1 .63 

Serial 

Position 

Four 

1 

0  .98 

2  .39 

1 .28 

2  .03 

2 

1.29 

1  .63 

1.32 

1  .88 

Combined  reversals  and  correct  responses  made  with 
and  without  knowledge  of  display  type  by  16 
subjects.  Each  attempting  to  recover  from  four 
subliminally  entered  unknown  attitudes  while  using 
each  of  three  types  of  attitude  presentation. 

Correct  Total 

Responses  Trials 
50  64 

61  64 

55  64 


3  27 


Display  Type  Reversed 

Responses 

Moving  Horizon  14 

Moving  Airplane  3 

Frequency  Separated  9 


motion. 


Study  One  evaluated  the  transfer  of  training  from 
simulator  to  aircraft  by  aircraft  checkflight  maneuvers 
and  trials  to  proficiency  among  four  groups  of  pilots: 
two  groups  who  received  simulator  training  plus 
aircraft  training  in  proportions  prescribed  by  the 
regular  syllabus,  one  group  who  received  both  aircraft 
and  simulator  training,  except  that  their  simulator 
training  did  not  include  the  final  phase  of  landing  to 
touchdown.  Although  the  graded  checkflight  scores 
found  all  groups  roughly  equal  in  performance,  the  two 
groups  who  had  not  received  landing  training  in  the 
simulator  (includes  the  "aircraft  only”  group)  took 
more  aircraft  trial  landings  to  attain  proficiency  then 
the  other  group. 

The  motion  versus  no-motion  study  was  confounded  by 
several  factors,  of  which  the  most  important  was  that 
some  no-motion  subjects  did  in  fact  receive  some 
training  with  platform  motion.  Nevertheless,  despite 
the  favor  shown  by  both  instructors  and  pilots  to  the 
platform  motion  feature,  there  were  few  practical 
differences  attributable  to  the  addition  of  platform 
motion  in  training  as  measured  by  the  simulator  and 
these  differences  washed  out  when  transition  was  made 
to  aircraft  flight. 

The  study  of  motion  sickness  revealed  little  incidence 
of  motion  sickness  either  with  or  without  platform 
motion. 

b.  Results: 

(1)  Key  Data: 


(a)  Study 

One  (Landing 

Training) : 

Simulator 

Aircraft  Landings 

Group 

Landings 

to  Proficiency 

C-l  ( N=  27 ) 

28 

17 

C-2  ( N=  3  9) 

28 

28 

C-3  ( N=10 ) 

0 

50 

C-4  (N=19 ) 

23  * 

37 

*  Trainer  frozen  or  waveoff  indicated  at 
Select  Land  Flap  position  in  the  landing 
pattern  (i.e.,  no  touchdown  simulations) 


(b)  Study  Two: 


Average  Trials  to  Proficiency 

With-Motion  No-Motion 

( N=39 )  ( N=ll ) 

Totals  for 

13  tasks  42.5  53.2 

(Study  data  shows  13  tasks  broken  down.  For 
most  tasks  there  are  small  and  in  some  cases 
significant  differences,  and  in  three  cases 
large  differences,  as  follows:) 

Abort  4  Engine  1.5  3.0 

Abort  3  Engine  2.9  4.9 

Holding  4.1  1.7 

(2)  Verbal  Description: 

(a)  Study  One:  The  Transfer  Effectiveness 
Ratios  computed  from  the  landing  data  show 
that  landing  practice  in  the  simulator 
provides  a  training  benefit  under  the  three 
different  training  conditions  examined. 

(b)  Study  Two:  Simulator  training  without 
cockpit  motion  versus  with  cockpit  motion 
shows  in  gene,  al  small  significant  but  not 
practical  benefits  from  cockpit  motion. 
Individual  differences  among  students  had 
more  effect  on  trials  to  proficiency  than  did 
training  method.  Later  trials  to  proficiency 
in  the  aircraft  for  four  and  three  engine 
aborts  were  not  significantly  affected  by 
lack  of  cockpit  motion  in  the  simulated 
training.  Both  students  and  instructors 
subjectively  strongly  favor  the  use  of 
platform  motion  in  the  simulator. 

c.  Authors'  Conclusions:  Study  results  indicate  that 
simulator  practice  in  landing  pattern  airwork  and  the 
final  phase  of  landing  transfers  positively  to  the 
aircraft.  This  transfer  occurs  even  though  instructor 
and  student  pilots  universally  agreed  that  the  2F87F 
does  not  "handle"  like  the  aircraft  during  the  final 
phase  of  landing.  Although  both  students  and 
instructors  strongly  favor  the  use  of  platform  motion 
in  the  simulator,  experimentally  it  was  found  that 
individual  differences  among  students  had  more  effect 
on  trials  to  proficiency  than  did  the  presence  or 
absence  of  simulated  cockpit  motion.  The  addition  of 


344 


platform  motion  does  significantly  improve  performance 
of  most  tasks,  but  not  with  practical  importance  except 
in  two  tasks,  three  and  four  engine  aborts.  These 
differences  washed  out  in  later  trails  in  the  aircraft. 


1.  Author:  Semple,  Clarence  A.  Jr. 


2.  Title:  Training  Effectiveness  Evaluation:  Device  1D23, 
Communication  and  Navigation  Trainer 

3.  Source:  NAVTRAEQ LJ I PCEN  72-C-0209-2,  March,  1974; 
National  Technical  Information  Service  AD  766  619 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Flight  Navigation  ;  Flight  Communications  ; 

Training  Effectiveness  . 

5.  Short  Summary:  An  evaluation  of  the  1D23  flight 
communications  and  navigation  training  device  finds  that  it 
enhances  navigation  skills  transferred  to  aircraft  in  many 
aspects,  which,  however,  can  only  be  detected  by 
supplementary  performance  criteria  outside  the  usual  overall 
4-point  flight  performance  grade. 

6.  Devices:  Training  aircraft  used  as  transfer  testing 
environments : 

a.  T-29  and  C-114 

b.  T-3  9 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando 
FL  32813 

b.  Performing  Organization: 

Manned  Systems  Sciences,  Inc. ,  Northridge,  CA 

8.  Type  of  Article:  Experiment  (four  separate  experiments) 

a.  Number  of  Groups:  2  groups  in  each  of  4 
experiments 

b.  Description  of  Groups:  In  each  experiment, 
subjects  and  controls  were  matched  by  aptitude  scores 
and  instructor  assessments  of  intangible  criteria 
(e.g.,  morale).  In  experiments  3  and  4  they  were  also 
balanced  by  progress  in  the  flight  navigation  training 
to  that  date 

(1)  Experiment  1:  23  subjects,  23  controls 

(2)  Experiment  2:  23  subjects,  23  controls  (same 

trainees  as  in  Experiment  1) 

(3)  Experiment  3:  15  subjects,  15  controls 

(4)  Experiment  4:  14  subjects,  14  controls 

c.  Tests  or  Trials/Timing:  In-process  trials  and 


post-tests  (checkf lights) 


d.  Number  of  Different  Types  of  Measures:  5 

e.  Description  of  Measurements  and  Ratings: 

(1)  Standard  4-point  Aviation  Training  Forms  and 
Grading  (all  experiments) 

(2)  Supplementary  Performance  Evaluation:  more 
objective,  task-specific  measures,  used  in 
Experiments  2  and  4 

(3)  Student  questionnaires 

(4)  Instructor  questionnaires 

(5)  Instructor  interviews 

f.  Experimental  Setting/Training  Context: 
institutional,  hands-on' 

g.  Statistical  Methods:  t-tests;  factorial  analysis 
of  variance;  multiple  discriminant  analysis 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Experiments  1  &  3:  T-29  and  C-114 

training  aircraft  (transfer  environment)  1D23 
training  device 

(b)  Experiments  2  &  4:  T-39  training 

aircraft  (transfer  environment)  1D23  training 
device 

(2)  Fidelity  Levels  of  1D23  trainer: 

(a)  Physical:  medium-high 

(b)  Functional:  medium-high 

(3)  Type  of  Task/Skill  Required:  operations; 
cognitive;  procedural;  whole-task 

(4)  Task  Difficulty: 

i.  Stage  of  Training:  introduction,  skill 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step  and 
self-  paced  (trainees  were  allowed  additional  practice 
on  the  device  on  their  own  time) 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  favorable  as  assessed  by 
questionnaire 

(2)  Students:  not  discussed 

m.  Use  of  Instructional  Features: 

( 1 )  Intensity :  assumed  intensive 

(2)  Features  used:  Sign-in  Capability; 
Number/Quality  of  Responses;  Automated 
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Demonstration;  perhaps  Internal  Monitoring  of 
Instructional  Features;  others  not  specified 

Abstract : 

a.  Study  Synopsis:  This  study  of  the  effectiveness  of 
jet  flight  communication  and  navigation  simulator 
Device  1D23  in  a  specific  syllabus  of  instruction  was 
separated  into  four  experiments  with  the  following 
obj  ectives : 

(1)  To  assess  impact  of  device  training  on  student 
performance  in  dead  reckoning  navigation 
performance  in  the  T-29  or  C-114  aircraft; 

(2)  Same  for  student  communication  and  navigation 
task  performance  in  the  T-39  training  phase; 

(3)  Same  for  increased  device  training  on  student 
performance  in  dead  reckoning  navigation 
performance ; 

(4)  Same  for  increased  device  training  on  student 
performance  in  the  T-39  training  phase. 

The  first  two  experiments  compared  the  aircraft 
(transfer)  performances  of  groups  that  had  received 
training  on  the  device  with  groups  that  had  not 
received  training  on  the  device.  The  last  two 
experiments  compared  the  aircraft  performance  of  groups 
that  received  the  standard  amount  of  device  training 
with  groups  that  received  an  additional  session  on  the 
device. 

Groups  in  each  experiment  were  closely  matched  by 
Flight  Aptitude  Ratings  and  by  Aviation  Qualification 
Test  scores,  by  more  subjective  factors  assessed  by 
instructors  (e.g.,  morale),  and,  in  the  later  training 
phases,  by  progress  in  the  syllabus  to  that  point  in 
the  training. 

Measurements  in  all  experiments  were  the  standard  4- 
point  Aviation  Training  Forms  and  Grading,  and  student 
questionnaires,  and  instructor  questionnaires  and 
interviews.  In  experiments  2  and  4,  a  supplementary 
task-  oriented  performance  measure,  judged  to  be  more 
objective,  was  added  to  supply  greater  evaluative 
sensitivity . 


Results : 


(1)  Key  Data  -  Experiment  2 

SUMMARY  OR  PERFORMANCE  IMPROVEMENTS  RESULTING 
FROM  TRAINER 

Mean  Value  Mean  Value  Chance 


Measure 

Untrained 

Group 

Trained 

G  roup 

Difference 

Probability 

No.  of  No.  2 
Needle  Reading 
Errors 

.58 

.09 

.001 

No.  of  Altimete 
Reading  Errors 

r 

.  81 

.30 

.001 

No.  of  Wrong 
Headings 

2.01 

.88 

.001 

No.  of  Fuel 
Mgmnt.  Errors 

.58 

.36 

.05 

Minutes  of  get 
on  Radial 

3.85 

2 .47 

.01 

Pet.  of  time 
on  Radial 

59.65 

78.37 

.001 

No.  of  Missed 
Calls 

2.38 

1.65 

.05 

ETAs,  No.  Min.s 
off 

1.52 

1.08 

.01 

No.  of  Voice 
Comm.  Errors 

3  .77 

2.86 

.  05 

No.  of  Turn 
Point  Errors 

1  .28 

1.06 

.05 

(2)  Verbal  Description:  In 

Experiments  3  and  4, 

no  significant  differences  in  performance  were 
detected  on  any  measures. 

Experiment  1  yielded  conflicting  results. 

Although  no  significant  differences  were  found 
between  groups  in  the  standard  4-point  scoring, 
instructor  opinion  strongly  supported  the 
effectiveness  of  the  device.  For  that  reason,  th 
supplementary  grading  criteria,  which  separated 
the  communications  and  navigation  tasks  into  13 
sub-tasks  with  more  objective  measurement 
parameters,  was  employed  in  Experiment  2  (and  4). 

The  Key  Data  for  Experiment  2  appear  in  the  table 
above,  which  shows  the  ten  measures  on  which  the 
device- trained  group  demonstrated  significant 
superiority  to  the  group  not  trained  on  the 
device.  Of  the  other  3  sub-task  measures,  no 


significant  differences  were  found. 

As  with  the  other  experiments,  however,  in 
Experiment  2  there  was  no  significant  difference 
in  standard  4-point  overall  instructor-scored 
flight  grades. 

Other  results  included  instructor  assessments  of 
device  capabilities  and  deficiencies;  the  most 
salient  points  being  the  absence  of  flight  stress 
and  criticality  in  the  simulator,  the  absence  of 
pilot-  pacing  of  activity  which  is  the  rule  in  the 
airplane,  and  the  lack  of  static  and 
communications  patter  which  are  typical  in  an 
airplane.  In  the  area  of  physical  fidelity,  a 
device  shortcoming  was  its  nearly  exclusive 
reliance  on  digital  indicators,  whereas  analog 
instrumentation  predominates  in  the  aircraft. 

c.  Author's  Conclusions:  It  is  felt  that  reliable 
conclusions  regarding  the  training  effectiveness  of 
Device  1D23  cannot  be  drawn  from  flight  grades  alone. 
The  strength  and  consistency  of  instructor  opinion  data 
•  support  the  conclusion  that  training  in  the  device  has 

resulted  in  improvements  in  virtually  every  aspect  of 
student  confidence  and  performance  in  the  execution  of 
dead  reckoning  tasks  in  an  airborne  setting;  and 
statistical  analysis  of  supplemental  measures  indicate 
that  training  on  the  device  has  resulted  in  numerous 
improvements  in  student  performance  during  T-39 
training  flights. 

Task  areas  for  which  moderate  or  no  improvement  in 
student  performance  was  observed  following  training  in 
the  device  can  be  considered  as  definitions  of  areas  in 
which  the  training  effectiveness  of  the  device  could  be 
further  enhanced. 

Additional  device  training,  beyond  that  already 
incorporated  into  the  syllabus,  had  no  effect  upon 
subsequent  aircraft  performance  in  either  dead 
reckoning  navigation  tasks  or  performance  of  airways 
navigation  and  communication  tasks. 
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1.  Authors:  Smith,  Russell  L. ,  Pence,  Gail  G. , 

Queen,  John  E. ,  &  Wulfeck,  Joseph  W. 

2.  Title:  Effect  of  a  Predictor  Instrument  on  Learning  to 
Land  a  Simulated  Jet  Trainer 

3.  Source:  National  Technical  Information  Service  AD/A-000 
586,  August  1974 

4.  Topic  Keywords:  Predictor  Instrument  ; 

Flight  Simulator  ;  Landing  Training  . 

5.  Short  Summary:  An  evaluation  of  a  glideslope  predictor 
instrument  as  a  training  aid  in  a  simulated  landing  task 
found  the  predictor  effective  in  improving  glideslope 
control  performance  in  the  transfer  task  in  the  simulator. 

6.  Devices: 

a.  Simulator  with  CRT  display  showing  2-dimensional 
aircraft  .glideslope  position,  attitude  control, 
throttle  control,  and  indicators  of  engine  power  and 
airspeed,  audio  stall  alarm,  no  other  controls  or 
displays 

b.  Computer-generated  analog  graphic  glideslope 
predictive  display  on  simulator  CRT 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research  , 
1400  Wilson  Blvd. ,  Arlington,  VA  22209 

b.  Performing  Organization:  Dunlap  and  Associates  , 
Inc.,  Western  Div.,  115  So.  Oak  St.,  Inglewood,  CA 
90301 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  10 

b.  Description  of  Groups: 

( 1 ) — ( 9 )  Subjects:  5  Air  Force  and  Navy  ROTC 
students  from  Los  Angeles  Area  ROTC  programs,  ages 
18-24,  little  or  no  flying  experience  but  met  A.F. 
or  Navy  qualifications  for  flight  training. 
Randomly  selected  except  balanced  between  groups 
for  flying  experience. 

(10)  Controls:  5  same 

c.  Tests  or  Trials/Timing:  In-process  trials 
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a.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Integrated  altitude  error 

(2)  Integrated  airspeed  error 

f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  ANOVA;  Duncan's  multiple- 
correlation  test;  rank-correlation 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above  section  6 

(2)  Fidelity  Levels  (for  simulator,  both 
predictor-aided  and  non-predictor-aided) : 

(a)  Physical:  medium-low 

(b)  Functional:  medium-low 

(3)  Type  of  Task/Skill  Required:  operations, 
cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  medium-high 

i.  Stage  of  Training:  not  applicable  (experimental) 

j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  not 
applicable 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  augmented  feedback; 
number/quality  of  responses 

Abstract : 

a.  Study  Synopsis:  The  primary  objective  of  this 
study  was  to  explore  a  predictor  instrument's  utility 
as  a  training  aid  in  a  relatively  complex,  psychomotor 
task.  The  task  was  to  control  the  glideslope  in  the 
simulated  landing  of  an  aircraft.  The  simulator 
consisted  of  (1)  a  CRT  display  of  continuous  aircraft 
vertical  and  forward-and-  backward  position  in  relation 
to  an  ideal  glidepath  line  drawn  on  the  CRT  face;  (2) 
an  engine  power  indicator;  (3)  an  airspeed  indicator; 

( 4 )  an  audio  stall  warning ;  and  (5)  throttle. 

The  predictor  used  in  the  experiment  was  a  computer¬ 
generated  trace  line  on  the  CRT,  which  projected  out 
from  the  instantaneous  aircraft  position  marker  and 
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showed  what  the  future  two-dimensional  glideslope  of 
the  aircraft  would  be  if  the  controls  were  to  be  held 
constant  from  that  instant.  In  the  experiment,  three 
different  predictive  time-spans  were  used  (illustrated 
by  shorter  or  longer  projected  lines  on  the  CRT  face) : 
5,  10,  and  20  seconds. 

A  second  experimental  variable  of  the  predictor  was  the 
"adaptive  mode",  which  signified  the  out-of-tolerance 
condition  under  which  the  predictive  trace-line  would 
appear  (under  ideal  operator  control  input,  no  trace 
line  appeared) .  Three  levels  of  adaption  were  employed 
in  this  study:  (1)  Adaptive  Mode  0,  in  which  the 
predictor  trace  appeared  only  when  the  instantaneous 
path  was  out  of  tolerance;  (2)  Adaptive  Mode  5,  in 
which  the  trace  appeared  whenever  the  patn  would  be  out 
of  tolerance  as  of  5  seconds  in  the  future;  (3) 
Adaptive  Mode  10,  in  which  the  trace  appeared  whenever 
the  path  would  be  out  of  tolerance. 

The  nine  (3  x  3)  experimental  conditions  thus  generated 
were  applied  to  nine  groups  of  subjects  consisting  of  5 
ROTC  volunteers  age  18-24  each,  who  had  little  or  no 
flight  experience  but  had  met  Air  Force  or  Navy  flight 
qualifications.  A  tenth,  control  condition  was  applied 
to  an  equivalent  group  of  5;  groups  were  randomly 
selected  except  balanced  by  flight  experience. 

All  training  and  testing  took  place  in  the  simulator. 
The  controls  did  not  receive  any  aid  from  the  predictor 
during  training  or  testing,  whereas  the  experimental 
subjects  received  predictor  aid  during  training  but  not 
during  testing.  Test  trials  were  alternated  with 
training  trials  throughout  the  experiment. 

Performance  measurements  on  the  tests  were  of 
integrated  altitude  error  (deviations  about  the  ideal 
glideslope),  and  integrated  airspeed  error. 

b.  Results: 


(1)  Key  Data: 

Mean  Altitude  Error  as  a  Function  of 
Prediction  Span  and  Adaptive  Mode: 


Prediction  Span  5 

sec.  - 

error 

287 

Prediction  Span  10 

sec.  - 

error 

263 

Prediction  Span  20 

sec .  - 

error 

218 

Adaptive  Mode  0 

-  error 

270 

Adaptive  Mode  5 

-  error 

269 

Adaptive  Mode  10 

-  error 

225 
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Reviewer's  Note:  Figures  are  approximations  from 
graphs 

(2)  Verbal  Description:  In  general,  with  the 
exception  of  Condition  2  (prediction  span  5  sec., 
adaptive  mode  5),  all  experimental  conditions 
appeared  to  be  superior  to  the  control  condition. 
Condition  9  (prediction  span  20  sec. ,  adaptive 
mode  10)  demonstrated  by  far  the  highest  level  of 
performance  at  the  start;  most  of  the 
experimental  conditions  showed  sharp  declines  in 
error  scores  during  the  first  24  trials. 

The  table  above  shows  the  effect  of  each  variable 
when  averaged  over  the  other  variable.  For  the 
variable  of  prediction  span,  an  ANOVA  yielded  an  F 
ratio  of  20.71,  significant  beyond  the  .01  level. 

A  Duncan's  range  test  revealed  that  performance  on 
each  of  the  three  prediction  spans  was 
significantly  different  (p  <.05)  from  the  others. 
Thus,  performance  improved  significantly  on  test 
trials  as  prediction  span  during  training 
increased  from  5  seconds  to  20  seconds. 

With  regard  to  adaptive  mode,  an  ANOVA  yielded  an 
F  ratio  of  11.56  which  was  again  significant 
beyond  the  .01  level.  A  Duncan's  range  test 
indicated  that  the  10  second  adaptive  mode  was 
significantly  better  than  the  0  and  5  modes 
( p <■ . 0  1 )  .  These  results  indicate,  that  the  sooner 
the  predictor  instrument  appeared  on  the 
operator's  display  during  training,  the  better  the 
performance  on  test  trials. 

A  rank  correlation  between  altitude  and  airspeed 
error  yielded  a  coefficient  of  .794.  The  results 
of  airspeed  error  were  similar  to  those  of 
altitude  error  with  respect  to  prediction  span  and 
adaptive  mode. 

c.  Authors’  Conclusions:  Not  all  predictor  displays 
led  to  higher  performance  than  the  control  condition. 

In  general,  conditions  having  the  shortest  prediction 
spans  and  adaptive  modes  exhibited  somewhat  poorer 
performance  than  the  control  condition  on  a  number  of 
trials,  while  the  opposite  configurations  showed 
consistently  superior  performance.  Perhaps  a 
meaningful  explanation  for  the  differential  results  is 
related  to  the  concept  of  "f acilitation-interf er ence . " 
Periodically  observing  the  predictor  trace  at  the 
expense  of  continuously  monitoring  the  aircraft  symbol 
might  have  conflicted  with  control  operations  and 
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degraded  performance  in  the  cases  where  the  predictor 
provided  only  minimal  forecasting  information  (that  is, 
when  it  appeared  more  rarely  and  gave  only  a  brief 
predictive  time-span) . 

In  the  present  study,  not  only  were  mean  performances 
of  several  predictor  groups  substantially  superior  to 
the  control  condition,  standard  deviations  were  also 
very  depressed,  indicating  very  narrow  distributions 
about  high  mean  performance  levels. 

Results  of  this  study  suggest  strongly  that  the 
predictor  instrument  may  have  considerable  utility  as  a 
training  aid  in  a  wide  variety  of  complex,  manual 
control  tasks.  Transfer  effects  appear  to  have 
achieved  practical,  as  well  as  statistical 
significance.  Our  data  suggest  that  a  more 
appropriately  designed  predictor  display  (with  a  span 
of  30  seconds  and  an  adaptive  mode  of  15  seconds,  for 
example)  would  have  led  to  an  even  greater  transfer 
effect  than  that  observed  in  this  study. 
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1.  Author:  Spangenberg,  Ronald 

2.  Title:  Tryout  of  a  General  Purpose  Simulator  in  an  Air 
National  Guard  Training  Environment 

3.  Source:  AFHRL-TR-7 4-92 ,  December  1974 

4.  Topic  Keywords:  Maintenance  Training  Simulator  ; 

General  Purpose  Simulator  ;  Technical  Training  ; 

Maintenance  Troubleshooting  ;  Training  Effectiveness  . 

5.  Short  Summary:  A  study  of  the  use  of  a  General  Purpose 
Simulator  for  cn-the-job  training  of  experienced  technicians 
found  it  highly  acceptable  to  trainees,  and  a  performance 
test  indicated  it  was  an  efficient  training  device. 

6.  Device:  EC  II  General  Purpose  Simulator  for  Maintenance 
Training;  simulating  APQ  Radar  System 


7.  Institutions: 


a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 


b.  Performing  Organization:  Technical  Training 
Division,  Air  Force  Human  Resources  Laboratory,  Lowry 
AFB,  CO  80230 


8.  Type  of  Article:  Experiment 


a.  Number  of  Groups:  2 
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f.  Experimental  Setting/Training  Context:  Field,  GJT 

g.  Statistical  Methods:  not  specified 
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c.  Authors'  Conclusions:  The  results  from  the  male 
subjects  gave  the  authors  support  for  their  hypotheses. 
The  negative  effect  of  overtraining  was  significant  in 
transfer,  and  there  was  a  nonsignificant  "trend"  toward 
higher  fidelity  (in  the  form  of  5  stacks  in  training 
rather  than  3  stacks)  also  having  some  negative  effect 
on  transfer. 

The  authors  conclude:  "since  overtraining  may  or  may 
not  be  advantageous  in  a  simulation  situation,  it  is 
important  then  to  determine  what  is  being  overtrained. 
It  would  appear  that  if  a  strategy  is  well  learned,  but 
not  the  specifics  of  the  training  device,  then  this  is 
not  detrimental  to  ultimate  performance." 

The  contradictory  findings  for  the  females  are 
partially  explained  by  the  hypothesis  that  different 
hand  sizes  between  males  and  females  may  have  led  to 
different  learning  strategies. 


<)• 


b. 


Resul ts : 


(1)  Key  Data: 

AVERAGE  TIME  (IN  SECONDS)  ON  TRANSFER  ON  1ST 
AND  5TH  TRIAL  BLOCK,  BY  TRAINING  CONDITION  &  SEX 


Block  1 

Block 

5 

8 

Racks 

(Control) 

Male 

83 

50 

- 

Female 

91 

59 

5 

Racks 

Overtrained 

Mai  e 

78 

60 

- 

Female 

77 

48 

5 

Racks 

Not  Overtra.  - 

Mai  e 

83 

56 

- 

Female 

81 

59 

3 

Racks 

Overtrained 

Mai  e 

81 

58 

- 

Femal e 

83 

51 

3 

Racks 

Not  Overtra.  - 

Mai  e 

76 

'  53 

- 

Female 

81 

53 

P 

h  .05 

for  superiority  of  male 

controls 

over 

male 

simulator-  trained  groups  by  Block  5 

(2)  Verbal  Description:  All  gr oups--exper imental 
and  control--showed  improvement  over  the  5  blocks 
of  transfer  trials.  The  control  groups  showed  an 
initial  inferiority  in  Trial  1  and  then  began  to  f 

catch  up. 

(a)  Among  male  subjects:  the  "overtrained" 
subjects  trained  on  5  racks  showed  the  least 
improvement  on  the  transfer  trials,  and 
performed  most  poorly  on  the  final  block. 

The  "not  overtrained"  group  trained  on  3 
racks  improved  the  most,  and  performed  best 
among  the  trained  groups  on  the  final  block. 

The  other  two  groups'  relative  performance 
also  accorded  with  the  hypotheses  propounded 
above;  the  "overtrained"  3  stack  group  was 
inferior  to  the  "not-overtrained"  5  rack 
group.  However,  all  the  experimental  groups 
were  outperformed  by  the  control  group,  which 
improved  by  far  the  most  over  the  5  transfer 
blocks,  and  finished  with  the  best 
performance  on  the  final  block. 

(b)  Among  female  subjects:  contr adictory 
trends  were  evident.  The  "overtrained" 

5-rack  group  improved  and  finished  the  best 
during  transfer;  the  "overtrained"  3-rack 
group  was  second  in  improvement  and  final 
performance;  the  control  group  lagged  far 
behind  with  the  "not-  overtrained"  5-stack 
group. 
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equipment.  These  overlearned  inappropriate  responses 
can  cause  negative  transfer.  The  authors  suggest  that 
criterion  performance  levels  during  training  be  used  to 
cut  off  training  at  the  optimal  point,  and  to  minimize 
overlearning  and  negative  transfer. 

Secondly,  aside  from  the  cost  disadvantages  of 
achieving  high  realism,  the  face  validity  of  a 
simulator  may  be  deceptive  as  long  as  it  is  not  100% 
identical  to  the  actual  equipment--which  by  definition 
it  cannot  be.  The  more  highly  realistic  a  device,  the 
more  misleading  any  inappropriate  elements  embedded  in 
it  may  be.  In  some  cases,  a  lower  fidelity  simulation, 
affording  a  more  generalized  response  during  learning, 
may  be  less  likely  to  overtrain  the  student  into  bad 
habits,  because  in  transferring  to  the  actual 
equipment,  he  or  she  will  less  likely  identify,  and 
thus  confuse,  aspects  of  the  simulator  with  aspects  of 
the  actual  equipment. 

An  experiment  in  a  simple  perceptual-motor  task  was 
used  to  demonstrate  these  principles.  Equal  numbers  of 
male  and  female  experimental  subjects  were  trained  to 
f)  •  perform  a  collating  operation  removing  sheets  of  paper 

from  stacks  in  a  machine.  In  full  operation  the 
machine  would  produce  8  stacks.  Subjects  were  trained 
by  collating  either  3  or  5  stacks,  and  tested  for 
transfer  by  collating  8  stacks.  Time  of  training  was 
also  a  factor,  in  that  each  group  was  subdivided  into 
"overtrained"  and  "not  overtrained"  groups  (the 
"overtrained"  were  given  additional  training  trials 
beyond  those  required  to  reach  a  criterion  performance 
level) . 

In  the  transfer  condition,  five  blocks  of  trials  were 
given.  A  control  group  which  had  been  given  no  prior 
training  in  the  task  also  performed  5  blocks  of  the 
transfer  task.  The  one  measure  taken  for  all  subjects 
was  time  to  complete  the  task. 
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g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  in  section  6  above; 

training  simulations  varied  as  follows: 

(a)  "Overtrained"  on  3  racks 

(b)  "Overtrained"  on  5  racks 

(c)  "Not  overtrained"  on  3  racks 

(d)  "Not  overtrained"  on  5  racks 

(2)  Fidelity  Levels:  3  Racks  5  Racks 

Physical:  Medium  High  Medium  High 

Functional:  Medium  High  Med. high  to 

High 

(3)  Type  of  Task/Skill  Required:  Collating; 
psychomotor;  perceptual;  part-task 

(4)  Task  Difficulty:  Low 

i.  Stage  of  Training:  introduction,  skill  (not 
strictly  applicable--exper imental  only) 

j.  Trainee  Sophistication:  Novice;  but  the  task  was 
so  simple  they  were  probably  expert  by  the  time 

experiment  was  completed  f 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 
(not  strictly  applicable--experimental  only) 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  applicable 
9.  Abstract: 

a.  Study  Synopsis:  This  paper  discusses  two  principal 
issues  in  the  optimization  of  simulator  training,  and 
describes  an  experiment  to  illustrate  the  hypotheses 
which  are  advanced. 

Central  to  the  discussion  is  the  distinction  between 
optimal  and  maximal,  especially  as  a  reflection  of  the 
possible  deterioration  of  performance  due  to  negative 
transfer  when  a  simulator  is  overused.  The 
common-sense  validity  of  designing  a  simulator  to 
resemble  actual  equipment  as  closely  as  possible,  and 
the  validity  of  training  students  on  the  simulator  as 
long  as  they  continue  to  learn  on  it,  are  both  brought 
into  question. 

First  of  all,  increasing  time  spent  learning  on  the 
simulator  tends  to  overtrain  all  responses,  including 
the  ones  that  may  be  inappropriate  on  the  actual 
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1.  Authors:  Weitz,  Joseph./  &  Adler,  Seymour 

2.  Title:  The  Optimal  Use  of  Simulation 

3.  Source:  Journal  of  Applied  Psychology,  1973,  58(2), 
219-224 

4.  Topic  Keywords:  Simulation  ;  Negative  transfer  ; 
Overtraining  ;  Simulator  Fidelity  . 

5.  Short  Summary:  Due  to  negative  transfer,  a  result  of 
overtraining,  the  optimal  amount  of  simulator  practice  and 
the  optimal  degree  of  simulator  fidelity  may  be  much  less 
than  is  usually  assumed  under  the  "more  is  better" 
principle. 

6.  Devices: 

a.  T-8  Thomas  table-top  collater  with  8  operational 
racks  (transfer  device) 

b.  Same  but  with  only  3  or  5  racks  operating  (training 
device) 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research  , 
Air  Force  Systems  Command 

b.  Performing  Organization:  Department  of  Psychology, 
New  York  University 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  10  (5  male,  5  female) 

b.  Description  of  Groups: 

( 1 ) — ( 4 )  Subjects:  10  males  each 

(5)  Control:  10  males 

( 6 )  — ( 9 )  Subjects:  10  females  each 
(10)  Controls:  10  females 

c.  Tests  or  Trials/Timing:  Post-test 

a.  Number  of  Different  Types  of  Measures:  1 

e.  Description  of  Measurements  and  Ratings:  Time  to 
complete  task 

f.  Experimental  Setting/Training  Context:  laboratory, 
hands-on 
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The  research  described  in  this  report  was  directed  to 
improving  the  laboratory  model,  maximizing  the 
transportability  of  the  software  to  new 
microprocessors,  and  augmenting  the  software  to  allow 
Navy  instructors  to  manage  the  system. 

Capabilities  of  the  device  are  enumerated  and  data  on 
the  system's  reliability  and  maintainability  are  also 
provided. 

b.  Results:  Results  of  informal  preliminary 
experimental  studies  with  advanced  prototypes  of  the 
GMTS  in  training  maintenance  troubleshooting  were 
generally  positive.  In  one  of  the  experiments  cited, 
students  trained  on  the  GMTS  were  field  tested  for 
transfer  on  the  actual  equipment  with  encouraging 

r esul ts . 

c.  Authors'  Conclusions:  The  GMTS  hardware  and 
software  have  been  refined  and  documented  to  meet  the 
research  objective  of  providing  the  intensive  training 
desired  at  low  cost.  Testing  and  evaluation  of  the 
device  in  the  school  environment  are  scheduled. 
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f.  Experimental  Setting/Training  Context:  field 
testing  in  a  naval  school  setting 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  GMTS  as  in  section  6  above 

(a)  Simulating  Fleet  communications  System,  a 
multi-equipment  system  for  radio 
communications 

(b)  Simulating  AN/APA-66  radar  repeater 

(2)  Fidelity  Levels: 

(a)  Physical:  Medium-low  (2  dimensional 
microfiche  projections  of  equipment) 

(b)  Functional:  Medium-high 

(3)  Type  of  Task/Skill  Required:  Maintenance 
troubleshooting;  cognitive,  psychomotor; 
whole-task,  part-task 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  not  specified 

j.  Trainee  Sophistication:  not  specified 

k.  Incorporation  of  Device  into  P.O.I.:  not  specified 

l.  User  Acceptance  or  Attitude:  not  specified 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  not  specified 

(2)  Features  used  (available):  Restart/Resequence 
Capability;  Malfunction  Selection;  Sign-in 
capability;  Number/Quality  of  Responses;  Next 
Activity  Features;  Automated  Demonstration 

9.  Abstract: 

a.  Study  Synopsis:  This  report  briefly  reviews  the 
development,  from  a  laboratory  model,  of  a  working 
Generalized  Maintenance  Training  simulator.  The  report 
details  hardware  and  software  features  of  the  device,  a 
computer-based  simulator  that  automatically  selects 
malfunctions  and  displays  high-resolution  color  images 
of  the  actual  maintenance  testing  equipment.  The 
student  can  rapidly  access  close-up  views  of  any 
section  of  the  equipment  (and  conversely,  "zoom  out" 
again)  and  can  interact  with  the  displayed  switches  by 
touching  desired  switch  settings  with  a  small  stylus. 

The  device  has  the  capability  to  simulate  a  wide 
variety  of  electronic  equipment  and  to  present  a 
multiplicity  of  maintenance  set-up,  systems  assessment, 
and  troubleshooting  problems  to  be  selected  by  the 
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1.  Authors:  Towne,  Douglas  M. ,  &  Munro,  Allen 

2.  Title:  Generalized  Maintenance  Trainer  Simulator: 
Development  of  Hardware  and  Software  -  Final  Report 

3.  Source:  NPRDC  TR  31-9,  April  1981 

4.  Topic  Keywords:  Maintenance  Training  Simulation  ; 
Troubleshooting  ;  Generalized  Simulator  . 

5.  Short  Summary:  A  review  of  preliminary  experimental 
studies  of  a  generalized  maintenance  training  simulator 
designed  to  provide  intensive  training  in  set-up, 
troubleshooting,  and  system  assessment  for  a  variety  of 
electronic  equipment.  Results  indicate  that  the  device  is 
ready  for  formal  testing  and  evaluation  in  a  Navy  school. 

6.  Device:  Generalized  Maintenance  Training  Simulator  ,  a 
computer-based  simulator  that  automatically  selects 
malfunctions  and  displays  high-resolution  color  images  of 
the  equipment.  Student  interacts  by  touching  with  a  small 
stylus. 

7.  Institutions: 

a.  sponsor:  Navy  Personnel  Research  and  Development 
Center,  San  Diego,  CA  92152 

b.  Performing  Organization:  Behavioral  Technology 
Laboratories,  University  of  Southern  California, 
Redondo  Beach,  CA  92152 

8.  Type  of  Article:  Experiment  (Two  informal  preliminary 
field  tests  are  ci ted--Rigney ,  Towne,  King,  &  Moran,  1978; 
and  Rigney,  Towne,  Moran,  &  Mishler,  1980 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  20  radio  communications  students 

(2)  Controls:  10  radar  students 

c.  Tests  or  Trials/Timing: 

(1)  35  problem  trials 

(2)  33  problem  trials 

d.  Number  of  Different  Types  of  Measures:  not 
specified 

e.  Description  of  Measurements  and  Ratings:  not 
specified 


The  simulator  was  of  low  physical  and  functional 
fidelity,  consisting  of  a  sheet  with  erasable  material 
covering  information  for  each  item.  The  examinee 
attempted  to  solve  the  problem  by  erasing  as  few  items 
as  possible  to  location  of  the  fault. 

b.  Results: 


(1)  Key  Data: 

Rank  Order  Inter-test  Correlations  for 
Five  Performance  Variables  (N=14) 


Test  Score  Category 

No.  of  Steps 

No.  of  Correct  Steps 

No.  of  Incorrect  Steps 

Total  time 

Tester's  Rating 

(2)  Verbal  Description: 
variables  were  found  to 
between  tests  (see  above 


Simulated  x  Actual 

-.32 

.14 

-.19 

-.50 

-.35 

Four  of  the  five  scored 
be  negatively  related 

)  . 


Overall  troubleshooting  strategy  was  apparently 
influenced  by  the  test  format  since  results  showed 
that  only  1  of  the  14  examinees  selected  the  same 
initial  troubleshooting  step  on  both  his  actual 
and  simulated  test. 


Correlations  were  also  made  between  these  test 
scores  and  the  overall  performance  achievement 
scores  of  students  for  the  equipment  phase  of  the 
same  course.  High  significant  correlations  were 
found  (.55)  between  the  actual  equipment  test 
score  and  the  overall  performance  score,  whereas 
the  correlation  with  the  simulated  test  and  the 
overall  performance  score  was  -.36. 

c.  Author's  Conclusions:  The  simulated  performance 
measure  employed  in  this  investigation  did  not  provide 
a  valid  estimate  of  proficiency  on  the  identical  task 
requiring  actual  performance.  The  evidence  strongly 
suggests  caution  in  replying  upon  simulated  tests,  even 
those  with  considerable  face  validity,  without 
empirical  validation. 
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(5)  tester's  rating; 

(6)  number  of  parts  replaced 


f.  Experimental  Setting/Training  Context: 
institutional,  hands-on 

g.  Statistical  Methods:  Rank  order  correlation 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above, 

(2)  Fidelity  Levels: 

Physical 
Functional 

(3)  Type  of  Task/Skill 
cognitive,  psychomotor, 

(4)  Task  Difficulty: 


Simulator 

Low 

Low 

Required 


section  6 
Actual  Equip. 
Highest 
Highest 
maintenance, 


perceptual , 
Medium-high 


par t- task 


i.  Stage  of  Training:  Skill 

j.  Trainee  Sophistication:  not  specified;  presumed 
intermediate 


k.  Incorporation  of  Device  into  P.O.I.:  lock-step 
(not  instructional) 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  applicable 
9.  Abstract: 


a.  Study  Synopsis:  As  part  of  a  larger  research 
effort  to  develop  a  practicable  system  for  validly 
assessing  performance  skills  in  various  Navy  technical 
ratings,  this  study  investigated  how  people  perform  on 
an  actual  equipment  performance  test  as  compared  with 
how  they  perform  on  the  identical  task  in  simulated 
performance  format. 

The  task  to  be  performed  was  a  troubleshooting  to 
locate  a  faulty  resistor  in  a  radio  receiver. 
Performance  measures  taken  included:  (1)  number  of 
steps;  (2)  number  of  correct  steps;  (3)  number  of 

incorrect  steps;  (4)  total  time;  (5)  tester's  rating 
and  (6)  number  of  parts  replaced.  14  students  in  an 
experimental  Naval  Electronics  Technician  training 
course  were  tested  with  both  actual  equipment  and  a 
simulator.  The  simulated  and  actual  equipment  tests 
were  administered  in  cour ter-balanced  order  with 
approximately  3  weeks  between  the  first  and  second 
testing. 
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1.  Author:  Steinemann,  John  H. 

2.  Title:  Comparison  of  Evaluations  Obtained  from 
Assessment  of  Proficiency  on  Simulated  and  on  Actual 
Performance  Tasks 

3.  Source:  Naval  Personnel  Research  Memo  SRM  67-1,  July 
1966 

4.  Topic  Keywords:  Electronics  Troubleshooting  ; 
Proficiency  Testing  ;  Simulation  ;  Low  Fidelity  . 

5.  Short  Summary:  A  comparison  of  a  paper  and  pencil 
simulation  of  a  troubleshooting  task  with  the  same  task 
performed  on  actual  equipment  showed  the  actual  equipment 
trial  superior  as  an  evaluator  of  proficiency. 

6.  Devices: 

a.  Simulator:  Paper  and  pencil  test  sheet  with 
covering  material  which  when  erased  reveals  information 
given  for  each  item. 

b.  Actual  Equipment:  superheterodyne  radio  receiver 

7.  Institutions: 

a.  Sponsor:  U.S.  Naval  Personnel  Reseaich  Activity, 
San  Diego,  C A 

b.  Performing  Organization: 

U.S.  Navy  Training  Research  Laboratory,  U.S.  Navy 
Personnel  Research  Activity,  San  Diego,  CA 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  1 

b.  Description  of  Groups:  Subjects:  14  students  in 
an  experimental  Navy  Electronic  Technicians  training 
course 

c.  Tests  or  Trials/Timing:  2  (1  test  on  each  device 
for  every  member  of  group,  in  counterbalanced  order) 

d.  Number  of  Different  Types  of  Measures:  6 

e.  Description  of  Measurements  and  Ratings: 

(1)  number  of  steps; 

(2)  number  of  correct  steps; 

(3)  number  of  incorrect  steps; 

(4)  total  time; 
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simulator  was  designed  to  achieve  were  accomplished  by 
all  personnel.  Motivational  similarity — a  feeling  or 
attitude  on  the  part  of  the  student  of  functional 
similarity  between  the  real  equipment  and  the 
simulator--  was  also  achieved.  Use  of  a  GPS  provides  a 
reasonably  economical  simulation  capability  when  a 
variety  of  simulations  are  required  in  a  training 
program  or  when  low  student  flow  permits  sharing  of  GPS 
capability  among  different  programs. 


evaluated  the  device.  Measures  of  test  performance 
were  number  of  parts  replaced,  and  time  to  isolate  the 
malfunction. 

*  Reviewer's  Note:  It  is  my  inference  that  the  testing 
was  done  on  the  GPS--author  failed  to  make  explicit. 

b.  Results: 


(1)  Key  Data:  Learning  On  The  GPS: 


Learning  Indicator 

Qualified 

U  nqual . 

Personnel 

Per  sonnel 

Did  you  learn  purpose  of  controls? 
Can  you  interpret  normal/malfunction 

(yes/ total) 

(yes/total) 

11/11 

6/6 

operation? 

11/11 

6/6 

Can  you  perform  system  self  checks? 

9/11 

6/6 

Can  you  isolate  malfunctions? 

10/11 

5/6 

Mean  number  replacements  (units) 
Median  of  average  malfunction 

1.43 

1.71 

isolation  time  (minutes) 

1 .76 

2.82 

(2)  Verbal  Description:  A  performance  test  showed 
a  high  level  of  achievement  using  the  GPS.  Of 
particular  note  is  the  programmed  capability 
provided  by  the  simulator  enabling  training  in  the 
isolation  of  selected  malfunctions.  Experienced 
mechanics  given  the  write-up  averaged  less  than 
two  minutes  to  identify  and  correct  each 
malfunction  in  the  simulation.  Time  to  clear  a 
malfunction  on  the  flight  line  would  average  over 
an  hour  per  malfunction  because  of  the  need  to 
actually  remove  the  faulty  component,  and  probably 
slightly  longer  when  training  is  also  being 
accomplished. 

The  recommendations  of  the  experienced  mechanics 
indicate  that  the  GPS  is  usable  for  field 
training.  Student  and  instructor  acceptance  of 
the  GPS  was  indicated  by  the  questionnaire  data; 
experienced  mechanics  generally  indicated  that 
training  could  be  performed  easier  and  faster  on 
the  GPS. 

c.  Author's  Conclusions:  This  report  indicates  the 
efficiency  and  effectiveness  of  the  simulation  of  an 
APQ  126  Radar  System  by  the  GPS  in  a  field  training 
environment.  Data  colleced  from  field  use  indicated 
that  psychological  realism  was  economically  captured  in 
the  simulation  of  the  APQ  126.  The  goals  which  the 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 

(2)  Fidelity  Levels: 

(a)  Physical:  medium-low 

(b)  Functional:  medium-low 

(3)  Type  of  Task/Skill  Required:  maintenance, 
troubleshooting,  cognitive,  psychomotor ,  part-task 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  Training:  Group  1,  skill;  Group  2, 
transition 

j.  Trainee  Sophistication:  intermediate  to  expert 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  highly  favorable  as  measured  by 
questionnaire 

(2)  Students:  same 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 

(2)  Features  used:  Augmented  feedback; 

Malfunction  selection;  number/quality  of 
responses;  others  not  specified 

9.  Abstract: 

a.  Study  Synopsis:  This  study  of  the  EC  II  General 
Purpose  Simulator  endeavored  to  evaluate  the 
acceptability  and  efficiency  of  the  device  in  providing 
on-the-job  skill  training  to  Air  National  Guard 
maintenance  technicians.  The  GPS  had  already  been 
shown  cost-effective  in  a  technical  training  resident 
school  environment.  The  GPS  has  great  program 
flexibility  to  simulate  a  wide  variety  of  equipment,  at 
the  sacrifice  of  physical  similarity  to  the  specific 
equipment  being  simulated.  Aspects  of  functional 
dissimilarity,  such  as  augmented  feedback  and  varying 
the  part-task  sequence,  help  to  make  the  GPS  superior 
to  actual  equipment  as  a  trainer.  In  this  study,  the 
GPS  was  used  to  simulate  the  APQ-126  Radar  System. 

Eleven  of  the  subjects  of  this  study  were  qualified  on 
the  APQ-126  system  being  simulated;  nine  were  not. 
Skill  levels  and  experience  were  generally  higher  among 
the  experienced  subjects. 

All  subjects  were  given  troubleshooting  training  on  the 
simulator,  tested  for  troubleshooting  isolation  on  the 
imulator,*  and  given  questionnaires  by  which  they 
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1.  Authors:  Weitzman,  Donald  0.,  Firieberg,  Michael  L. , 
Gade,  Paul  A.,  &  Compton,  George  L. 

2.  Title:  Proficiency  Maintenance  and  Assessment  in  an 
Instrument  Flight  Simulator 

3.  Source:  Human  Factors  ,  1979,  21(6),  701-710 

4.  Topic  Keywords:  Instrument  Flight  ;  High  Fidelity  ; 
Proficiency  Maintenance  . 

5.  Short  Summary:  An  instrument  flight  training  high- 
fidelity  simulator  is  shown  effective  in  maintaining  and 
even  improving  the  instrument  flight  proficiency  of 
experienced  Army  helicopter  pilots. 

6.  Devices:  High  Fidelity  Helicopter  Instrument  Flight 
Simulator  2B24 

7.  Institutions: 

a.  Sponsor:  Deputy  Chief  of  Staff  for  Operations, 
Aviation  Offi-ce,  Department  of  the  Army 

b.  Perforin'1  ng  Organization:  Army  Research  Institute  , 
Alexandria  VA  22333,  and  Aviation  Group,  101st 
Airborne  Div.,  Ft.  Campbell,  KY 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  3 

b.  Description  of  Groups: 

(1)  Subjects:  Two  groups  of  12  each  fully 
qualified  combat-ready  Army  helicopter  pilots 

(2)  Controls:  One  group  of  12,  same 

c.  Tests  or  Trials/Timing:  Pre-test  and  Multiple 
Post- tests 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Performance  Measures  (ratings  by 
Instructor-Pilot  observers) : 

(a)  Checkrides  in  Aircraft, 

Instructor-scored,  scored  50%  by  procedural 
knowledge  and  50%  by  flight  control 
performance:  three  post-tests  in  addition  to 

"base-line"  pretest 

(2)  Supervisor  Ratings:  None  except  as  implied  by 
scoring  of  Performance  Checkrides 
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(3)  Pilot  questionnaires 

f.  Experimental  Setting/Training  Context: 

Experimental 

g.  Statistical  Methods:  analysis  of  variance; 

Tukey's  HSD  test;  trend  analyses;  product-moment 
correlation  coefficient;  regression  methods 

h.  Variables  Being  Manipulated: 

(1)  Training  Device: 

Helicopter  Instrument  Flight  Simulator  2B24 

(2)  Fidelity  Levels:  High 

(3)  Type  of  Task/Skill  Required:  Instrument 
Flight  of  UH-1H  helicopter;  perceptual, 
psychomotor ,  part-task 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Maintenance  of  skills 

j.  Trainee  Sophistication:  High  (qualified  pilots) 

k.  Incorporation  of  Device  into  P.O.I.:  Lock-step 

l.  User  Acceptance  or  Attitude:  Very  good  as  measured 
by  questionnaire 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  Not  specified,  presumably 
intensive 

(2)  Features  used:  Not  specified 
9.  Abstract: 

a.  Study  Synopsis:  Evidence  from  training  of 
commercial  aircraft  pilots  indicated  that  instrument 
flight  simulation  training  can  help  maintain 
real-aircraft  instrument  flight  proficiency.  In  the 
present  study,  an  experiment  was  conducted  over  a 
9-month  period  with  experienced  Army  helicopter  pilots 
to  test  the  transfer  of  simulation  training  to  real 
aircraft  performance,  and  additionally  to  determine  the 
effectiveness  of  testing  on  the  simulator  to  assess 
instrument  flight  proficiency. 

Equal  numbers  of  high  skill  and  low  skill  experienced 
pilots  were  diviaed  equally  into  three  groups  of  12 
members  each:  (1)  those  to  be  trained  solely  in  the 

high  fidelity  instrument  flight  simulator  2B24;  (2) 

those  to  be  trained  solely  on  the  aircraft,  UH-1H 
helicopter;  (3)  those  to  receive  a  mix  of  training 
between  simulator  and  aircraft.  Each  group  was  tested 
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in  checkrides  on  the  simulator  before  and  at  the 
conclusion  of  the  experimental  training  period;  and  on 
the  real  aircraft  before,  twice  during,  and  once  at  the 
conclusion  of  the  experimental  training  period.  All 
pilots  also  answered  a  questionnaire  eliciting  their 
judgement  as  to  the  effectiveness  of  the  training. 

The  results  appeared  to  support  the  hypothesis,  that 
simulator  training  could  maintain  and  even  improve  the 
instrument  flight  performance  among  experienced  pilots. 
In  fact,  the  simulator- trained  pilots  performed 
significantly  (though  only  slightly)  better  than  the 
aircraf t-trained  pilots  on  the  checkrides  in  the 
aircraft.  Furthermore,  the  improvement  of  initially 
low-  skill  pilots  during  the  training  brought  them 
nearly  level  with  the  high-skill  pilots  on  the  final 
checkr ide. 

The  high  correlation  between  final  aircraft  checkride 
performance  and  the  final  simulator  checkride 
performance  among  all  pilots  implies  the  value  of  the 
simulation  testing  in  assessing  the  instrument  flight 
proficiency  of  experienced  pilots. 

b.  Results: 

(1)  Key  Data: 

INFLIGHT  CHECKRIDE  PERFORMANCE  SCORES 

BY  SKILL  LEVEL  AND  TRAINING  CONDITION 


Basel ine 

1st 

2nd 

3rd 

Low  Skill 
Device  2B24 
Low  Skill 

58 

68 

80 

79 

Combined 

Low  Skill 

61 

72 

73 

75 

Inflight 

62 

61 

70 

70 

High  Skill 
Device  2B24 

81 

79 

83 

81 

High  Skill 
Combined 

80 

79 

80 

7  9 

High  Skill 
Inflight 

87 

6  9 

75 

80 

Simulator- trai ned  advantage  over  inf 1 ight- tr ai ned 
is  significant  at  p  0.05 


Combined-trained  not  significantly  different  from 
either  of  other  two  training  conditions  (;.  0.05) 


(2)  Verbal  Description:  Simulator  training 
maintained  and  in  some  cases  improved 
real-aircraft  checkride  performance  of  pilots, 
particularly  of  those  initially  less  skilled  in 
instrument  flight.  Checkride  performances  of 
simulator- trained  pilots  was  superior  to  that  of 
ai rcraf t- trained  pilots  in  the  tested  areas  of 
instrument  flight.  There  was  high  correlation 
between  checkride  performance  in  the  simulator  and 
checkride  performance  in  the  aircraft. 
Questionnaires  revealed  high  acceptance  of  the 
simulator  among  those  who  received  solely 
simulator  training. 

c.  Authors'  Conclusions:  The  results  are  consistent 
with  the  record  of  commercial  aviation  which  has  shown 
that  high  fidelity  simulators  can  provide  an  effective 
means  of  providing  training  for  highly  qualified 
pilots.  However,  it  should  be  emphasized  that  the 
effectiveness  of  any  training  device  depends  upon  how 
it  is  used....  The  evidence  of  transfer  in  the  present 
study  may  well  be  attributed  to  the  training  program 
rather  than  simulator  design  alone.  It  is  possible... 
that  if  inflight  training  is  conducted  with  as  much 
structure  and  control  as  can  occur  with  simulator 
training,  some  evidence  of  additional  benefits  might  be 
found. 
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1.  Authors:  Wheaton,  George  R. ,  &  Mirabella,  Angelo 

2.  Title:  Effects  of  Task  Index  Variations  on  Training 
Effectiveness  Criteria 

3.  Source:  Technical  Report:  NAVTRAEQUIPCEN  71-C-0059-1, 
October  1972 

4.  Topic  Keywords:  Task  Analysis  ; 

Quantitative  Task  Indices  ;  Training  Effectiveness  ; 
Transfer  of  Training  . 

5.  Short  Summary:  This  study  provides  some  validation  of  a 
set  of  quantitative  task  indices  as  predictors  of 
performance  of  an  operational  task,  in  terms  of  speed  and 
accuracy . 

6.  Devices:  Synthetic  sonar  trainer  in  three  basic 
configurations,  according  to  the  number  of  controls  and 
indicators  presented:  simple,  intermediate,  and  complex, 
plus  nine  additional  sub-configurations. 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orland, 
FL  32813 

b.  Performing  Organization: 

American  Institutes  for  Research 

8.  Type  of  Article:  Experiment  and  Field  evaluation 

a.  Number  of  Groups:  12 

b.  Description  of  Groups: 

(1)  Subjects:  5  paid  male  university  students 
from  Washington,  DC  metropolitan  area 

(2)  —  (12)  5  each,  same 

c.  Tests  or  Trials/Timing:  15  in-process  trials,  plus 
10  post-trials  for  5  of  12  groups.  Field  evaluation 
consisted  of  structured  interviews  of  experienced 
instructors. ) 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  time  to  perform  task, 

(2)  errors  of  omission, 

(3)  errors  of  commission, 

(4)  (for  Field  Evaluation):  estimates  provided  by 
experienced  instructors  in  ratio  form. 
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f.  Experimental  Setting/Training  Context:  Laboratory, 
hands-on 

g.  Statistical  Methods:  ANOVA;  single  variable 
regression  analysis;  product-moment  correlation 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above.  Additionally, 
device  training  trials  were  varied  according  to 
whether  every  feedback  light  worked,  none  worked, 
or  every  third  one  worked. 

(2)  Fidelity  Levels: 

(a)  Physical:  medium  to  high  (varied) 

(b)  Functional:  low  to  high  (varied) 

(3)  Type  of  Task/Skill  Required:  operations; 
psychomotor;  perceptual;  procedural;  part-task 

(4)  Task  Difficulty:  medium 

i.  Stage  of  Training:  introduction,  familiarization, 
skill,  experimental 

j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  not 
appl icable 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  specified 
9 .  Abstract : 

a.  Study  Synopsis:  The  purpose  of  this  study  was  to 
determine  the  relationships  between  systematic 
variations  in  quantitative  task  characteristic  indices 
were  developed  (e.g.,  "LV--the  link  value  reflecting 
the  relative  strength  of  the  sequence  of  use  among  the 
various  controls  and  displays.  As  used  here  it  is  the 
sum  of  the  products  of  the  number  of  times  a  link  is 
used,  and  the  percentage  of  use  of  the  link.")  A 
simulated  sonar  training  device  was  used  in  12 
different  configurations  ranging  from  simple  to 
complex,  which  provided  variation  in  all  the  indices. 
Twelve  groups  of  five  randomly  selected  university 
students,  one  group  for  each  configuration,  performed 
15  trials  of  a  set-up  procedure  on  the  training 
equipment.  Measures  of  time  taken  to  complete  the 
task,  and  number  of  errors  of  omission  and  commission 
were  recorded  during  these  trials.  5  of  the  12  groups 
proceeded  subsequently  to  take  a  transfer  test  in  which 
the  same  measures  were  taken. 
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A  second  part  of  the  study  evaluated  the 
appropriateness  of  the  task  index  criteria  via 
structured  interviews  with  experienced  sonar 
instructors  in  the  field. 

b.  Results: 


(1)  Key  Data: 
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INDEX 

VALUES 
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INTERCORRELATIONS  OF  TASK  INDEX  VALUES  AND  MEAN 
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81 
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46 

61 

53 

69 

17 
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58 

32 

34 

46 

43 

78 

14 
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.576,  p 
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.05 

degrees 
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.  01 

r  = 

MAIN  -  the  number  of  responses  comprising  the  main  or 
dominant  procedural  sequence  in  an  operations  flow  chart. 

CNTG  -  the  number  of  responses  comparisir.g  the  auxiliary  or 
contingency  procedural  sequences. 

TA  -  the  total  number  of  responses  (actions)  comprising  the 
procedural  sequence  in  an  operations  flow  chart.  It  is  the 
sum  of  MAIN  and  CNTG. 

CONT  -  the  total  number  of  different  controls  manipulated 
during  performance  of  a  subtask. 

DISP  -  the  total  number  of  different  displays  referenced 
during  the  performance  of  a  subtask. 

(2)  Verbal  Description:  Variations  in  performance 
due  to  type  of  task  were  clearly  seen. 

Correlation  of  task  indices  with  mean  performance 
time  at  each  trial  block  were  in  general  highly 
consistent  (see  tables  above).  It  was  found  that 
the  strength  of  correlation  between  task  indices 
and  performance  grew  with  the  number  of 
trials — i.e.,  as  learning  increased,  the 
correlation  became  stronger. 

In  the  transfer  task,  it  was  found  that  error  was 
proportional  to  the  distance  (along  a  similarity 
dimension)  between  interpolated  and  transfer 
tasks. 

The  findings  in  the  field,  through  interviews  with 
instructor  personnel,  tended  to  correlate  well 
with  the  laboratory  data. 

c.  Authors'  Conclusions:  Complete  statistical 
analysis  of  the  results  showed  that  the  multivariate 
approach  is  essential--i . e . ,  individual  task  indices 
did  not  appear  capable  of  predicting  performance  on  the 
training  devices.  Rather,  collections  of  indices,  with 
perhaps  specific,  but  as  yet  unidentified  patterns  of 
features,  are  crucial.  Moreover,  there  is  some  hint  in 
the  results  that  these  patterns  may  depend  upon 
training  stage. 

In  addition  to  implying  that  predictor  patterns  may 
vary  with  stage  of  training,  the  results  also  imply 
that  criterion  patterns  may  be  simularly  influenced. 
Thus,  the  designer  may  have  to  ask,  not  whether  indices 
relate  to  training  effectiveness,  but  what  patterns  of 
indices  relate  to  what  criterion  of  effectiveness  at 
what  stage  of  training.  These  are  questions  which  the 
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present  research  cannot  answer. 

Results  from  the  transfer  trials  suggest  that  operators 
trained  on  synthetic  devices,  when  transferred  to  field 
devices,  might  pay  a  price  in  speed,  which  is  not 
readily  mitigated,  although  conceivably  they  could 
attain  a  satisfactory  level  of  accuracy. 

The  current  research  effort  has  supported  the 
feasibility  of  relating  quantitative  indices  of 
equipment  design  to  performance,  at  least  for  the 
restricted  set  of  indices  and  trainer  stacks  examined 
in  the  present  study.  The  effort  has  also  supported 
the  feasibility  of  predicting  transfer  effects  from 
equipment  design  indices. 
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(  '  Generic  Indices  consisted  of  29  separate 
measures  drawn  from  several  domains.  The  two 
most  important  domains  of  indices  developed 
were  Siegel's  Display  Evaluative  Index 
(DEI) — a  dimensionless  measure  of  the 
effectiveness  with  which  information  flows 
from  displays  via  the  operator  to 
corresponding  contr ols--and  the  Panel  Lay-Out 
and  Task  Type  Indices  (Fowler  et.al.,  1968), 
which  can  vary  independently  of  the  DEI. 

Seven  other  generic  indices  were  also 
developed. 

(b)  Specific  Indices;  within  this  set  were 
indicies  specific  to  surveillance  trainers 
and  to  certain  sub-tasks  within  those 
trainers. 

(2)  Device  Task  Description  and  Differentiation. 
Indices  were  applied  to  three  Naval  surface  sonar 
trainers  on  four  training  sub-tasks:  set-up, 
detection,  localization,  and  classification  of 
targets. 

(3)  The  predictive  method  was  an  adaption  of  the 
standard  multiple  regression  model.  Mean  task 
scores  replaced  the  usual  individual  criterion 
scores,  and  quantitative  task  index  values  were 
used  as  predictor  scores. 

(4)  Predictions  were  applied  to  21  studies  (22 
experiments)  in  the  literature.  The  DEI  and  the 
Panel  Lay-Out  Index  could  not  be  applied  to  these 
studies.  However,  five  other  task  characteristics 
rating  scales  (out  of  a  total  of  18  that  were 
analyzed)  were  used  to  predict  the  criterion 
performance  of  pe rcent-time-on-target  (%  T.O.T.) 
after  5,  10,  and  15-minute  intervals  of  practice 
(the  percent  time-on- tar get  was  a  measure  common 
to  all  the  experiments  under  study) . 


b.  Results: 

(1)  Key  Data  (from  the  post-directive  prediction 
phase)  : 

Multiple  Correlations,  Performance  to  Task  Indices 
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(2)  Verbal  Description: 

(a)  Relevant  to  the  Device  Task  Description 
and  Differentiation  phase,  need  for  revision 
and  refinement  of  most  of  the  indices  became 
apparent.  The  specific  indices,  in 
particular,  seemed  inconclusive  or  to  be  of 
limited  value.  Several  of  the  miscellaneous 
generic  indices  were  easy  to  apply  and 
generated  data  of  some  interest.  However, 
the  Display  Evaluation  Index  appeared  highly 
useful,  as  it  possessed  diagnostic  value  and 
was  intuitively  satisfying,  varying  in 
accordance  with  subjective  impressions  of 
sub-task  difficulty  (for  example,  it  gave  a 
low  value  for  the  object  classification 
sub-task,  which  reflected  the  poor 
classification  performance  which  has  been 
reported  in  sonar  surveillance  (Levy  & 
Mirabella,  1968) . 

(b)  Post-dictive  predictions  showed  some  high 
correlations  between  trainee  performance  a 
combined  set  of  five  indices  when  applied  to 
an  early  stage  of  training.  The  table  Set  1 
above  shows  the  multiple  correlations  of 
these  indices  and  trainee  percent  time  on 
target  after  5,  10,  and  15  minutes  of 
training;  the  table  shows  both  a  decrease  in 
R  and  a  decrease  in  significance  of  the  data 
as  training  progressed. 

Another  combined  set,  shown  as  Set  2  above, 
showed  stronger  correlations  at  interim 
stages  of  training,  though  its  correlation  at 
the  initial  stage  was  lower. 

c.  Authors'  Conclusions:  An  overall  appraisal  of  the 
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findings  of  the  regression  analyses  indicates  that  the 
criterion  measures  used  in  the  post-diction  studies  are 
of  two  distinct  types.  Initial  level  of  performance, 
the  first  type,  appears  to  be  predicted  most 
efficiently  by  descriptors  which  relate  to  features  of 
the  task  per  se .  At  initial  levels  of  performance,  the 
training  variables  used,  in  the  studies  have  had  little 
if  any  impact.  Dominant  factors  at  this  stage  are 
probably  aspects  of  the  task  itself  and  the  abilities 
of  the  subjects.  It  is  conceivable  that  the  bulk  of 
the  residual  variance  in  the  initial  performance 
predictions  resides  in  the  subject  factor. 

At  interim  training  levels,  predictive  efficiency  of 
the  majority  of  the  predictors  declines.  A  potential 
explanation  for  this  would  be  the  increasing  impact  of 
whatever  training  variables  are  in  effect  plus  the 
interaction  of  these  variables  with  subject 
characteristics. 

The  problems  and  limitations  of  the  post-diction  are 
many  and  should  not  be  slighted.  The  attrition 
experienced  as  the  search  went  on  for  suitable  studies 
placed  a  decided  limitation  on  how  far  the  results  of 
the  regression  analysis  may  be  generalized. 

The  development  should  proceed  in  three  directions. 
First,  refinement  of  the  ratings  scales  must  be 
undertaken.  Second,  attention  must  be  given  to 
development  of  training  technique  indices;  such 
indices  may  aid  in  the  prediction  of  advanced  levels  of 
proficiency.  Third,  the  types  of  indices  employed  in 
the  present  study  must  be  applied  to  actual  training 
devices  for  which  performance  criteria  are  available. 

This  study  has  demonstrated  the  feasibility  of  using  a 
variety  of  quantitative  indices  to  describe  salient 
characteristics  of  the  trainee  sub-tasks  found  in 
surveillance  system  training  devices. 

Results  of  the  post-diction  study  were  encouraging, 
being  obtained  in  spite  of  the  fact  that  the  major 
indices  of  interest  (DEI)  and  panel  lay-out  could  not 
be  employed,  and  that  differences  between  groups  of 
subjects  (a  violation  of  the  predictive  model)  could 
not  be  avoided. 
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5.  Short  Summary:  An  evaluation  of  three  different 
training  devices  that  simulate  the  bur st-on-tar get  tank  gun 
firing  task  finds  the  three  roughly  equal  in  effectiveness, 
although  differing  in  complexity,  physical  fidelity,  and 
handling  characteristics.  The  relative  superiority  of  the 
17-4M  on  some  indices  may  be  due  to  the  role  of 
instrtuctor s . 

6 .  Dev  ices : 

a.  (Control  Group  Practice  Firing  and  Transfer  Testing 

Device)  :  3A102B  laser  device  in  M60A1  tank 

b.  17-4  "Green  Hornet",  a  relatively  simple 
Burst-on-Target  simulator 

c.  17-4M,  a  version  of  the  17-4  modified  for  the  study 
to  improve  versatility,  instructional  capability,  and 
ease  of  handling  controls 

d.  17-B4  Conduct-of -F i r e  Trainer,  a  complex  and  more 
physically  realistic  device 
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a.  'lumber  of  Groups:  4 

c.  Description  of  Groups: 

(1)  Subjects:  18  tank  gunnery  trainees  in  Army 
Advanced  Individual  Training 
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Rating  Method  Evaluation  ;  Laws  of  Learning  ; 
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5.  Short  Summary:  The  author  questions  the  usefulness  of 
conventional  transfer  of  training  studies  and  pilot  ratings 
in  the  evaluation  of  flight  simulators.  He  proposes  that 
scientific  psychological  principles,  coupled  with  knowledge 
of  the  success  of  predecessor  devices,  may  provide  more 
satisfactory  evaluative  tools. 

6.  Devices:  none  uescribed.  Studies  are  cited  in  which 
the  School  Lir\<  '  :ainer  and  the  SNJ  simulator  and  other 
flight  training  devices  were  used. 

7.  Institutions: 

a .  Sponsor : 

University  of  Illinois  at  Champagne-Urbana 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  In  this  theoretical  paper,  a  sharp  critique 
is  made  of  the  two  main  methods  of  evaluating  flight 
training  simul ator s--the  transfer  of  training  experiment  and 
the  pilot  rating  method--and  an  alternative  method  of 
evaluation,  based  on  the  success  of  predecessor  devices  and 
the  laws  derived  from  scientific  research,  is  proposed. 

Three  principal  flaws  in  the  transfer  of  training  method  are 
identified.  (1)  The  first  is  cost,  in  terms  of  simulator, 
parent  aircraft,  research  staff,  technicians,  and  subjects. 
The  other  two  flaws  pertain  to  the  requirement  that  both 
experimental  subjects  and  controls,  by  the  time  they 
transfer  to  the  parent  aircraft,  must  have  sufficient 
proficiency  to  avoid  accidents  and  generate  meaningful 
performance  measures.  Because  of  this,  (2)  the  controls  do 
not  present  a  no-practice  baseline,  and  (3)  the  experimental 
subjects  may  only  perform  under  the  precondition  of  a 
positive  outcome. 

Eight  major  problems  are  identified  in  the  pilot  rating 
method  of  evaluating  simulators:  (1)  The  assumption  that 

the  amount  of  transfer  of  training  is  positively  related  to 


A  part-task  simulator  is  defined  as  a  training  device  for 
only  critical  and  difficult  portion  of  the  flying  task.  It 
is  noted  that,  in  view  of  the  inconclusive  evidence  to  prove 
that  critical  part-task  trainer,  the  Air  Force  policy  was  to 
avoid  detailed  segmentation  of  task  into  low-order  elements. 

Four  examples  cf  part-task  trainers  are  adduced  and  design 
problems  are  discussed.  The  chief  problem  in  design  of  a 
part-task  simulator  is  how  much  to  simulate:  there  is 
little  evidence  to  guide  judgements  on  what  to  include  in  a 
part-  task  simulator. 

A  considerable  number  of  empirical  tr ansf er-of-training 
studies  are  cited  in  favor  of  the  effectiveness  cf  low  or 
moderate  rather  than  high  fidelity  in  the  areas  of  control 
response  precision,  visual  stimulus  noise,  proprioceptive 
feedback,  and  control-display  relations.  In  many  of  these 
studies,  differences  in  subject  performances  during  learning 
in  the  simulator  were  observed  as  result  of  varying 
fidelity;  however,  the  performance  differences  quickly 
disappeared  in  the  transfer  environment. 

It  is  important  to  note  that  these  findings  come  from 
studies  of  tasks  of  a  continuous  psychomotor  nature  and  that 
safe  generalizations  are  limited  to  this  class  of  activity. 
It  was  found,  for  example,  that  delay  of  feedback  on  the 
simulator  impairs  transfer  to  operational  equipment  where 
feedback  is  rapid  (though  transfer  is  not  impaired  in  the 
reverse  situation). 

Simulation  for  proficiency  measurement,  however,  appears  to 
require  higher  fidelity  than  training  simulation; 
furthermore,  proficiency  measurement  simulators  differ  from 
training  simulators  in  a  need  for  higher  reliability  and  for 
automatic  scoring  mechanisms,  and  the.  non-necessity  of 
familiarization  features  often  found  in  training  simulators. 
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relation  to  both  training  requirements  and  proficiency 
measurement  requirements.  Evidence  is  cited  to  show 
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Advantages  and  disadvantages  of  whole-task  simulators  are 
discussed,  and  the  criteria  for  effective  simulation  are 
applied  to  char acter i st i cs  of  a  specific  whole-task 
simulator,  the  F-86D,  which  at  the  time  of  this  paper's 
publication  was  one  of  the  more  sophisticated  trainers  in 
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on  the  average  older  and  had  spent  more  time  in  grade 
than  the  experimental  group. 

Subjects  and  controls  took  alternate  forms  of  a  40- 
item  multiple  choice  exam  immediately  before  and  after 
the  course  of  instruction;  they  also  took  an  oral, 
examiner-scored  performance  test  within  5  days  of  the 
end  of  the  course. 

Mul tipi e- r eg r ession  analysis  showed  that  OJT  time  prior 
to  the  course  was  a  significant  variable. 

b.  Results: 

(1)  Key  Data : 

OJT  TIME  Performance  Test  Score  Difference 


3  months 

MTU 

SIM 

Between  Groups 

or  less 

59.92 

69.66 

10.14I  O-  .  10  ) 

3-6  months 

6  months  or 

79.25 

74.55 

4.70 

more 

92.09 

89.18 

2  .91 

(2)  Verbal  Description:  No  statistically 
v  •  significant  difference  was  found  in  the 

performance  of  the  simulator-trained  and  the 
MTU-trained  groups  related  to  training  device. 
However,  when  the  subjects  and  controls  were 
subdivided  into  groups  in  respect  to  OJT  time  they 
had  received  prior  to  the  course  of  instruction,  a 
significant  difference  was  found  between  the 
groups  of  less-experienced  trainees  (under  3 
months  OJT)  on  their  performance  test  scores. 

c.  Authors'  Conclusions:  Instruction  on  the  simulator 
was  found  at  least  equal  to  instruction  on  the  MTU,  and 
there  is  evidence  favoring  the  benefits  of  the 
simulator  training  for  trainees  with  less  than  3  months 
OJT.  Since  the  T-2  aircraft  is  comparatively  simple, 
caution  must  be  exercised  in  extending  the  results  of 
this  study  to  more  complex  aircraft  systems. 
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multiple-  choice 

(2)  Performance  post-test,  an  examiner-scored  oral 
exam 

f.  Experimental  Setting/Training  Context: 
Institutional,  classroom  and  hands-on 

g.  Statistical  Methods:  Pooled  variance  "t"  test; 
multiple  linear  regression 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  EC  II  computerized  multiple  programmable 
maintenance  simulator 

(b)  Maintenance  Trainer  Unit  (actual 
equipment  modified  for  training) 

(2)  Fidelity  Levels: 

(a)  Physical:  of  simulator,  not  specified, 
apparently  medium-low;  of  MTU,  high 

(b)  Functional:  of  both  devices,  high 

(3)  Type  of  Task/Skill  Required:  part 
identification;  troubleshooting;  maintenance; 
cognitive;  psychomotor;  motor;  perceptual; 
procedural 

(4)  Task  Difficulty:  Medium 

i.  Stage  of  Training:  Skill,  transition 

j.  Trainee  Sophistication:  varied,  primarily 
intermediate 

k.  Incorporation  of  Device  into  P.O.I.: 
instructor-managed 

l.  User  Acceptance  or  Attitude:  not  discussed 

m.  Use  of  Instructional  Features:  not  specified 


9 .  Abstract: 


a.  Study  Synopsis:  The  study  was  designed  to  compare 
the  effectiveness  of  a  computerized, 

mul tiply-programmable  maintenance  training  simulator, 
the  EC  II,  with  a  conventional  hardware  trainer,  the 
Maintenance  Training  Unit,  or  MTU,  in  the  training  of 
maintenance  skills,  within  a  relatively  simple, 
one-week  maximum  course  of  instruction  (the  T-2C 
aircraft  maintenance  training  program;.  The 
experimental,  simulator-trained  group  consisted  of  35, 
the  control  group  of  87  naval  maintenance  personnel, 
having  a  fairly  wide  range  of  military  maintenance 
experience.  The  control  croup  trained  on  the  MTU,  was 
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1.  Authors:  Wright,  Joanne,  &  Campbell,  Jane 

2.  Title:  Evaluation  of  the  EC  II  Progammable  Maintenance 
Simulator  in  T-2C  Organizational  Maintenance  Training 

3.  Source:  NADC-75083-40 ,  May  1975 

4.  Topic  Keywords:  Maintenance-training-simulator  ; 
Aircraft  Maintenance  ;  Maintenance  Troubleshooting  . 

5.  Short  Summary:  A  study  evaluating  the  EC  II 
computerized  mul tiply-programmable  simulator  for  maintenance 
training  found  it  equally  as  effective  as  the  standard 
Maintenance  Training  Unit  which  was  a  piece  of  actual 
equipment  modified  for  training  purposes.  However,  the 
simulator  was  more  effective  for  trainees  with  less 
experience. 

6 .  Devices : 

a.  EC  II  computer ized . multiply  programmable 
maintenance  simulator 

b.  Maintenance  Training  Unit  (actual  equipment 
modified  for  training) 

7.  Institutions: 

a.  Sponsor:  Naval  Air  Systems  Command,  Washington,  DC 
20361 

b.  Performing  Organization: 

Naval  Air  Development  Center,  Warminster,  PA  18974 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  35  naval  maintenance  personnel 

(2)  Controls:  87  same,  but  with  average  2  years 
more  time  in  grade  than  subjects 

NOTE:  Subjects  and  controls  assigned  from  a  pool 

with  paygrade  levels  E-2  to  E-7  and  OJT  experience 
range  from  less  than  1  week  to  2  years. 

c.  Tests  or  Trials/Timing:  Pre-Test;  Post-Test 

d.  Number  of  Different  Types  of  Measures:  2 

e.  Description  of  Measurements  and  Ratings: 

(1)  Pre-test  and  Post-tests,  written:  40-item 
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favoring  the  ASUPT-  trained  groups  in: 

(a)  training  time  to  criterion  (see  above) ; 

(b)  contact  check  ride  scores  (see  above) ; 

(c)  grades  in  subsequent  training  block. 

No  significant  differences  were  found  between  the 
ASUPT- trained  subgroups  with  motion  and  without 
motion;  however  the  small  size  of  the  sample 
dictates  this  results  should  be  regarded  with 
caution. 

Questionnaire  results  showed  the  IP's  in  general 
favored  the  ASUPT  greatly  over  the  T-4. 

c.  Authors'  Conclusions:  The  study  has  clearly  shown 
that  a  sophisticated  full  mission  flight  simulator  can 
be  used  to  increase  training  effectiveness  in  the  Air 
Force  Undergraduate  Flying  Training  Program.  This  test 
realized  a  reduction  of  one-quarter  of  the  regularly 
scheduled  flying  training  hours  at  a  cost  of  only  2 
simulator  hours  for  each  T-37  aircraft  hours.  Savings 
in  instrument  training  has  precedent,  but  the  savings 
in  basic  contact  training  indicates  an  unexploited 
capability . 

While  sample  sizes  were  too  small  for  high  confidence 
conclusions,  there  was  no  evidence  in  this  study  that 
platform  motion  in  the  simulator  provided  an  increase 
in  transfer  of  training. 


(2)  Features  used:  unspecified.  For  device 
capability,  refer  to  AFHRL-TR-7 4-43 ,  pp.  17-26 

9.  Abstract: 

a.  Study  Synopsis:  This  study  was  designed  to  explore 
the  utility  of  integrating  the  Advanced  Simulator  for 
Undergraduate  Pilot  Training  with  a  new  training 
syllabus,  in  place  of  the  syllabus  using  the  older 
instrument  flight  simulator  T-4.  This  was  the  first 
effort  to  incorporate  a  full  mission  simulator  into  an 
operational  pilot  training  program.  Several  problem 
areas  were  identified  in  the  training  program  as  well 
as  in  the  device  itself. 

Eight  undergraduate  pilot  training  students  were 
trained  to  specified  levels  of  performance  in  all  major 
areas  of  basic  pilot  training  using  the  ASUPT;  half 
were  trained  using  the  platform  motion  system  and  half 
without.  Subsequently,  they  completed  basic  pilot 
training  in  T-37  aircraft.  Training  hours  required  and 
check  ride  scores  were  compiled  for  each  subject. 
Similar  data  were  collected  for  a  control  group  of 
eight  trainees  using  the  conventional  syllabus 
employing  the  T-4  instrument  trainer.  Outside  of  the 
device  training,  other  aspects  of  the  two  courses  in 
instruction,  such  as  time  in  classroom,  were  roughly 
equivalent . 

b.  Results: 

(1)  Key  Data: 

Aircraft  flying  hours  used  by  experimental  &  control  groups 
to  checkflight  proficiency;  group  means 


Basic  & 

Advanced 

Instru¬ 

Forma¬ 

Navi¬ 

* 

Presolo 

Contact 

ments 

tion 

gation 

Total 

AS  U  PT  Group 
Normal-Syllabus 

14.2 

18.7 

9  .0 

13.7 

7  .1 

70.7 

(T-4)  Group 

25.7 

19.4 

14.4 

14.8 

8  .2 

91.3 

%  hours  saved 
Training  Effec¬ 

4  5 

0  4 

3  8 

1  3 

1  3 

23 

tiveness  Ratio 

0.60 

0  .11 

0  .52 

1  .00 

0  .24 

0  .48 

*  (Note:  there  were  6  other  segments  for  which  data  were 
compiled  to  reach  the  total;  those  not  shown  here  as  no 
%  saved  or  T.E.R.  were  computed) 

Contact  Check  Rides:  Experimental  ASUPT  Group,  90.85%, 
Control  T-4  Group,  87.35%.  pH. 01 

(2)  Verbal  Description:  Significant  and 
practically  important  differences  were  found 


c.  Tests  or  Trials/Timing:  One  post-test 
(checkf light) ;  attitude  questionnaire  for  IP's 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Number  of  flying  hours  to  reach  criterion  (OK 
for  checkflight) 

(2)  Checkflight  scores  (IP  graded) 

(3)  Subsequent  performance  in  later  training  block 

f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 

g.  Statistical  Methods:  not  specified 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices: 

(a)  Advanced  Simulator  for  Undergraduate 
Pilot  Training,  high  fidelity  visual  and 
instrument  flight  simulator;  with  platform 
motion  and  without  motion 

(b)  T-4  Instrument  Trainer  used  in  standard 
syllabus 

(2)  Fidelity  Levels: 

(a)  Physical:  of  ASUPT,  unspecified, 
presumably  high  of  T-4,  unspecified, 
presumably  high 

(b)  Functional:  of  ASUPT,  high  of  T-4, 
unspecified,  presumably  medium-high  for 
instrument,  none  for  visual 

(3)  Type  of  Task/Skill  Required:  Aircraft 
piloting;  operations,  cognitive,  psychomotor, 
motor,  perceptual,  procedural,  whole  task 
(full-mission  jet  aircraft  flight) 

(4)  Task  Difficulty:  high 

i.  Stage  of  Training:  skill,  transition 

j.  Trainee  Sophistication:  intermediate 

k.  Incorporation  of  Device  into  P.O.I.: 
instructor-managed 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  very  good  as  measured  by 
questionnai re 

(2)  Students:  good 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  intensive 
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1.  Authors:  Woodruff,  Robert  R. ,  Smith,  James  F. , 

Fuller,  John  R. ,  &  Weyer,  Douglas  C. 

2.  Title:  Full  Mission  Simulation  in  Undergraduate  Pilot 

Training:  An  Exploratory  Study 

3.  Source:  AFHRL-TR-76-84 ,  December  1976 

4.  Topic  Keywords:  Aircraft  Flight  Simulation  ; 

Motion  Simulation  ;  Visual  Simulation  ; 

Training  Effectiveness  ;  Transfer  of  Training  . 

5.  Short  Summary:  An  exploratory  study  of  the  training 
effectiveness  of  the  Advanced  Simulator  for  Undergraduate 
Pilot  Training.  The  study  found  that  training  on  the  device 
can  reduce  the  trainees'  actual  aircraft  flight  requirements 
to  achieve  full  mission  proficiency,  as  compared  with  the 
conventional  syllabus  using  the  T-4  instrument  trainer. 

There  was  no  evidence  that  platform  motion  in  the  simulator 
produced  any  increase  in  transfer  of  training. 

6.  Devices: 

a.  Advanced  Simulator  for  Undergraduate  Pilot  Training 
high  fidelity  instrument  and  visual  flight  trainer 

b.  T-4  instrument  flight  simulator 

c.  T-37  jet  trainer  (transfer  environment) 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
AFHRL ,  Williams  AFB,  AZ  85224 

8.  Type  of  Article:  Experiment 

a.  Number  of  Groups:  2 

b.  Description  of  Groups: 

(1)  Subjects:  8  Undergraduate  Pilot  Students  with 
less  than  50  hours  flying  experience  (except  that 

1  had  approximately  200  hours  light  plane 
experience;  to  be  trained  on  ASUPT 

(a)  4  with  platform  motion 

(b)  4  without  motion 

(2)  Controls:  8  same  (all  with  less  than  50  hours 
flight  experiment) 


performance  decreased  as  the  transfer  practice 
continued. 

Acceptance  data  from  trainees  before  transfer  and  after 
training  indicated  a  significant  difference  in 
preferences  among  the  three  devices.  After  testing, 
however,  these  differences  disappeared.  Instructors 
consistently  rated  the  17-4M  better  than  the  other  two 
devices. 

c.  Authors'  Conclusions:  Detailed  analysis  indicated 
that  the  relative  superiority  of  the  17-4M  group  in 
some  aspects  of  transfer  can  most  likely  be  attributed 
to  the  role  of  instructors,  in  enhancing  training  on  a 
new--and  as  they  saw  it,  impr oved--device .  Several 
factors  in  addition  to  the  device  itself  can  determine 
what  skill  is  acquired,  how  rapidly  acquisition  takes 
place,  and  what  degree  of  transfer  results.  In  the 
present  study  instructors  appear  to  have  played  a 
central  role  in  determining  the  rate  of  acquisition  as 
well  as  the  nature  of  the  specific  skills  which  were 
acquired. 

The  three  devices  employed  in  the  BOT  transfer  of 
training  study  were  quite  similar  in  effectiveness. 

The  real  issue  in  evaluating  these  or  any  other  devices 
lies  in  defining  the  training  objective,  both  in  terms 
of  content  and  the  level  of  proficiency  required.  In 
the  stage  of  training  under  study,  the  emphasis  among 
training  personnel  appeared  to  be  on  ensuring  that 
trainees  know  how  to  apply  BOT,  and  not  on 
demonstration  of  a  high  level  of  proficiency.  Given 
this  objective,  use  of  the  17-B4  for  this  purpose  would 
seem  unwarranted.  The  17-B4  is  clearly  far  more 
sophisticated  than  is  necessary  in  this  context. 


9.  Abstract: 


a.  Study  Synopsis:  This  report  reports  on  an 
experiment  comparing  the  effectiveness  of  three 
Burst-on-Target  (BOT)  training  devices  for  preparing 
Advanced  Individual  Training  personnel  to  apply  BOT 
techniques  with  the  3A102B  laser  device  mounted  in  the 
M60A1  tank.  The  three  devices  were:  (1)  17-4  "Green 
Hornet",  a  relatively  simple  Burst-on-Target  simulator; 
(2)  17-4M,  a  version  of  the  17-4  modified  for  the  study 
to  improve  versatility,  instructional  capability,  and 
ease-  of  handling  the  controls;  (3)  17-B4, 

Conduct-of-Fi r e  Trainer,  a  complex  and  more  physically 
realistic  device.  Functional  fidelity  varied  between 
the  three,  but  not  in  a  way  systematic  enough  to  place 
the  three  devices  on  a  continuum. 

Three  groups  of  18  trainees  each  were  trained  to  a 
proficiency  criterion  on  the  devices.  A  fourth, 
control  group,  trained  for  a  specified  number  of  trials 
rather  than  to  criterion;  they  were  trained  on  the 
M3A102B  device  which  was  used  for  transfer  testing  of 
all  subjects.  Following  training,  all  groups  performed 
«  80  trials  on  the  M3A102B  laser  device--the  transfer 

test.  Accuracy  (percentage  of  hits),  speed  (time 
between  bursts)  data,  and  number  of  trials  to  criterion 
were  all  recorded.  In  addition,  device  acceptance 
questionnaire  data  were  obtained  from  both  trainees  and 
instructors. 

b.  Results:  In  the  acquisition  phase,  the  mean  of  the 
number  of  trials  required  to  reach  the  criterion  (90% 
hits  on  second  shot  of  the  BOT  series)  varied 
significantly  between  groups.  The  means  were  125.0  for 
the  17.4  group,  169.4  for  the  17. 4M  group,  and  134.1 
for  the  17. 4B  group. 

Speed  data  during  transfer  testing  showed  significant 
superiority  of  training  groups  over  the  control  group 
at  each  of  the  initial  four  blocks  of  trials,  which 
disappeared  thereafter.  It  appeared  that  the  fact  of 
having  had  prior  training  was  more  important  than  was 
the  nature  of  the  specific  device  on  which  practice 
occurred.  However,  the  accuracy  data  indicated  that 
prior  training  in  applying  BOT  did  not  have  a 
pronounced  effect  on  BOT  accuracy  during  transfer. 

A  second  set  of  effects  showed  that,  among  the 
simulator- trained  groups,  the  trainees  who  had 
practiced  on  the  17. B4  device  showed  no  significant 
improvement  in  speed  during  transfer,  while  those 
trained  on  the  other  two  devices  did.  The  spread  in 
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(2)  18  same 

(3)  18  same 

(4)  Controls:  18  same 

c.  Tests  or  Trials/Timing:  In-process  trials; 
post-test  (transfer  task) 

d.  Number  of  Different  Types  of  Measures:  3 

e.  Description  of  Measurements  and  Ratings: 

(1)  Percent  hits  (accuracy) 

(2)  Time  between  bursts  (speed) 

(3)  Trials  to  cr iter ion--in-process  trials  only 

f.  Experimental  Setting/Training  Context: 
Institutional,  hands-on 

g.  Statistical  Methods:  ANOVA;  Scheffe  tests; 
Vincent  curves 


h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  as  above,  section  6 

(2)  Fidelity  Levels: 


17-4 

Physical  Medium-low 
Functional  Medium 


17-4M 

Medium-low 

Medium 


17-B4 

Medium-high 

Medium 


(3)  Type  of  Task/Skill  Required:  operations, 
psychomotor,  perceptual,  part-task 

(4)  Task  Difficulty:  Medium-low 


i.  Stage  of  Training:  introduction,  familiarization, 
skill 


j.  Trainee  Sophistication:  novice 

k.  Incorporation  of  Device  into  P.O.I.:  lock-step 

l.  User  Acceptance  or  Attitude: 

(1)  Instructors:  more  favorable  for  the  17-4M 
device 

(2)  Students:  more  favorable  for  the  17-4M  device 
prior  to  transfer;  afterwards,  roughly  equal  for 
all  devices 

m.  Use  of  Instructional  Features: 

(1)  Intensity:  assumec  intensive 

(2)  Features  used:  not  specified 
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the  rated  similarity  between  simulator  and  aircraft  is  only 
a  partial  truth,  as  numerous  experiments  demonstrating  the 
effectiveness  of  low  fidelity  trainers  have  indicated.  (2) 
There  is  evidence  that  pilot  ratings,  which  are  subjective 
and  individual,  can  reflect  pilot  experience  level  as  much 
or  more  than  adequacy  of  simulation.  (3)  Experience  in  the 
simulator  can  make  the  simulator  rather  than  the  aircraft 
the  frame  of  reference.  (4)  Dimensions  of  the  simulator 
interact  and  may  not  always  be  distinguished  by  the  pilot. 

(5)  The  pilot  cannot  always  tell  where  the  poor  source  of 
simulation  originates,  nor  what  deficiencies  might  be  due  to 
his  own  lack  of  skill,  as  distinct  from  deficiencies  of  the 
device.  (6)  There  is  not  always  a  positive  correlation 
between  ratings  and  flying  performance.  (7)  Simulator 
effectiveness  cannot  be  separated  from  the  influence  of  the 
instructor  and  the  effectiveness  of  the  training  syllabus. 

(8)  Most  fundamentally,  the  flight  simulator  cannot  be 
regarded  as  an  "earthbound  aircraft";  rather,  it  is  a 
teaching  device,  and  must  be  judged  on  its  merits  as  a 
trainer,  not  on  its  faithfulness  of  mimicry. 

With  a  view  to  formulating  a  new  basis  for  flight  simulator 
evaluation,  five  major  psychological  principles  are 
prepounded;  (1)  the  dependence  of  human  learning  on 
knowledge  of  results;  (2)  that  the  key  to  piloting  is 
perceptual  learning,  which  is  an  increase  ir  the  ability  to 
extract  information  from  stimulus  patterns  as  a  result  of 
experience;  (3)  stimulus-response  learning  is  highly 
useful;  (4)  transfer  of  training  is  highest  when  similarity 
of  the  training  and  transfer  situations  is  the  highest;  (5) 
a  trainee  must  be  motivated,  and  the  task  itself  supplies 
some  of  the  motivation. 

The  author  states  that  "the  reason  for  putting  forth  these 
principles  is  the  assertion  that  a  system  built  on  sound 
scientific  laws  needs  less  concern  with  evaluation  because  a 
gooe  scientific  law  produces  accurate  prediction,  and  when 
the  outcome  can  be  predicted  it  is  redundant  to  conduct  an 
evaluation,"  and  concludes  that  there  is  a  "possibility  that 
system  testing  can  be  set  aside  when  the  scientific  laws  are 
known  to  be  strong  and  when  predecessors,  based  on  the  same 
laws,  have  been  successful." 
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pilot  tra  ning  covers  a  wide  range  of  topics  relating  to 
fidelity.  It  points  out  the  shortcomings  of  using  fidelity- 
alone  as  a  criterion  for  simulator  design,  and  recommends 
avenues  of  research  to  study  and  define  the  relationship  of 
fidelity  to  training  effectiveness. 

r«  f 

'  6.  Devices:  not  applicable 

7  .  Institutions : 

a.  Sponsor:  Advisory  Group  for  Aerospace  Research  and 
Development,  7  Rue  Ancelle,  9220  Nevilly-sur-Seine , 
France 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 

9.  Abstract:  The  AGARD  "Working  Group  10"  comprised 
engineers,  psychologists,  and  physiologists  from  the  U.S., 
Great  Britain,  Germany,  and  The  Netherlands.  In  this 
report,  the  team  attempted  to  "1.  Outline  a  framework  for 
the  logical  selection  of  training  simulator  facilities  with 
guidance  for  making  the  various  trade-offs.  2.  Address  the 
question  of  how  much  fidelity  is  required  to  train  a  given 
flight  phase  in  isolation." 

In  the  first  section  of  this  report,  the  general  term 
fidelity  is  separated  into  two  dimensions:  (1)  Objective 

Fidelity  "provides  an  engineering  viewpoint  and  is  the 
degree  to  which  a  simulator  would  be  observed  to  reproduce 
its  real-  life  counterpart  aircraft  in  flight,  if  its  form, 
substance  and  behavior  were  sensed  and  recorded  by  a 
nonphysiological  instrumentation  system  . "  (2) 
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Perceptual  Fidelity  "provides  a  psychological/physiological 
viewpoint  and  is  the  degree  to  which  the  trainee 
subjectively  perceives  the  simulator  to  reproduce  its 
real-life  counterpart  . " 

These  definitions  serve  to  distinguish  between  the  real  cues 
measured  objectively  and  what  the  pilot  experiences 
subjectively.  This  distinction  implies  one  important 
justification  for  reducing  engineering,  i.e.,  "objective" 
fidelity  in  the  simulator,  since  less  than  100%  objective 
fidelity  can  still  produce  100%  perceptual  fidelity. 
Unfortunately,  "the  knowledge  of  human  physiology  is 
insufficient  to  determine  how  much  objective  fidelity  is 
required  to  achieve  a  given  level  of  perceptual  fidelity." 

It  is  around  this  last  dilemma  that  the  bulk  of  this  report 
pivots.  The  following  sections  of  the  report  present  (1) 
the  training  specialists'  viewpoint  on  fidelity  of 
simulation;  (2)  an  overview  of  pilot  cuing  mechanisms, 
focussing  on  visual,  vestibular,  and  kinetic  motion  cues; 

(3)  the  simulator  technologists'  assessments  of  existing 
motion,  visual,  and  computer  model  technology;  and  (4)  the 
Working  Group's  overall  conclusions  and  recommendations. 
Appendices  provide  a  description  and  evaluation  of  methods 
of  assessing  training  effectiveness  (including  the  Simulator 
Fidelity  Model);  physical  characteristics  of  existing 
facilities;  a  survey  of  pilot  opinions  of  existing 
simulator  facilities;  a  detailed  review  of  the  technology 
of  visual  systems. 

Major  findings  and  conclusions  of  the  Working  Group  were  as 
follows: 

a.  Fidelity: 

(1)  Specific  cues  should  be  simulated  only  if 
essential  to  accomplishing  the  training  objective; 
this  necessitates  a  clear  statement  of  the 
training  objective. 

(2)  Visual  cuing  is  more  important  to  transfer 
than  motion  cuing,  given  current  technology. 

(3)  Simple  devices  can  be  very  effective  in  early 
stages  of  learning,  particularly  in  learning 
procedural  tasks. 

(4)  Reducing  the  level  of  complexity  in  the 
simulator  can  often  improve  training  effectiveness 
by  enabling  the  student  to  concentrate  on  those 
elements  relevant  to  the  task  being  trained. 
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(5)  Emphasis  should  be  on  achieving  perceptual 
fidelity  rather  than  objective  fidelity. 

(6)  "The  state  of  knowledge  of  sensory  information 
is  not  sufficient  to  completely  define  the  needs 
for  simulation.  Some  of  the  material...  may 
exist  in  the  perception  literature  in  forms  which 
could  be  made  useful  to  the  simulator  community." 

(7)  Further  research  is  required  "to  define  how 
cuing  ficelity  impacts  on  training  effectiveness 
for  a  matrix  of  aircraft,  tasks,  pilot  experience, 
and  instruction  techniques." 

(8)  One  approach  to  evaluating  the  effectiveness 
of  lower  fidelity  would  be  to  reduce  the  cuing 
fidelity  of  existing  devices  that  do  train  well, 
and  measure  the  effects,  rather  than  to  proceed  in 
the  opposite  direction  of  adding  fidelity  to 
simple  devices. 

b.  Instructional  Methods: 

(1)  The  most  effective  simulator  training  allows 
the  student  to  use  as  much  time  as  he  needs  to 
meet  a  criterion  standard  of  performance,  rather 
than  using  a  fixed  amount  of  time. 

(2)  Since  students  tend  to  reflect  instructor 
attitudes,  the  instructor  should  be  led  to  realize 
and  espouse  the  usefulness  of  the  device. 

(3)  Experienced  trainees  are  strongly  motivated  by 
competition  or  a  comparative  measure  of 
probability  of  success. 

c.  Assessment  of  Training  Effectiveness  -  is  best 
accomplished  through  the  transfer  of  training  model. 
"...Equating  training  effectiveness  with  fidelity  is  a 
coverup  for  our  ignorance  about  transfer  and  leads  to 
the  development  of  possibly  unnecessarily  costing 
devices. " 
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1.  Authors:  Baer,  Donald  K. ,  Jones,  D.  W. ,  & 

Francis,  Christopher  C. 

2.  Title:  Nuclear  Power  Plant  Simulators:  Their  Use  in 
Operator  Traini.no  anc  Requal if icati on 

3.  Source:  National  Technical  Information  Service 
NUREG/ CR-  1482,  July  1980 

4.  Topic  Keywords:  Nuclear  Power  Plant  Simulation  ; 
Comprehensive  Risk  Assessment  ;  Simulated  Malfunction  . 

5.  Short  Summary:  A  study  of  simulator  training  programs 
for  nuclear  power  plant  operators  in  the  U.S.  recommended 
ways  to  improve  the  training  programs,  princip.-lly  through 
developing  a  method  to  identify  what  malfunctions  are 
important  to  simulate. 

6.  Devices:  PWR  and  BWR  Nuclear  Power  Plant  Simulators 

7.  Institutions: 

a.  Sponsor:  Nuclear  Regulatory  Commission, 

Washington,  DC 

b.  Performing  Organization: 

Oak  Ridge  National  Laboratory,  TN  37830 

8.  Type  of  Article:  Survey 

9.  Abstract:  In  the  wake  of  the  1979  accident  at  Three 
Mile  Island,  the  Nuclear  Regulatory  Commission  ordered  a 
survey  of  the  capability  of  U.S.  nuclear  power  plant 
simulators  and  their  associated  operator  training  programs. 
This  study  conducted  by  the  Center  for  Nuclear  Studies  at 
Memphis  State  University  used  NRC  records,  Simulator  Bid 
Specifications,  simulator  development  studies,  simulator 
training  staff  interviews,  site  visits,  plant 
specifications,  licensee  event  reports  (LERs),  and 
nonnuclear  simulator  data,  as  evidence  for  this  report.  A 
capsule  review  of  the  TMI-2  accident  is  included  in  the 
report  to  illustrate  the  complexity  of  events  for  which 
operators  should  be  prepared.  The  subsequent  sections  of 
the  report  ceal  with  (1)  existing  nuclear  power  plant 
simulator  capabil iti as;  (2)  simulator  training  programs; 

(3',  proposed  procedures  for  selecting  equipment  malfunctions 
for  simulation. 

With  regard  to  existing  simulator  capabilities,  the  study 
concluded  that  in  general  "the  simulator  hardware  and 
software  are  not  the  limiting  factors  to  more  effective 
training...  rather,  it  is  the  utilization  of  simulators  in 
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training  programs. .. that  (could)  be  substantially  improved." 
It  was  noted,  however,  that  mathematical  models  of  two  phase 
coolant  flow  ( i . e . ,  water  plus  steam)  in  a  PWR  system  were 
still  inadequate  to  thoroughly  simulate  an  accident 
condition  such  as  transpired  at  Three  Mile  Island. 

In  view  of  the  general  satisf actor ipess  of  the  simulators 
themselves,  the  study  focussed  on  the  training  programs  and 
objectives.  Two  principal  issues  were  noted: 

(1)  The  NRC  "had  not  had  a  well-defined  regulatory 
framework  for  design  qualifications  oj  review  of 
simulators  or  their  use  in  operator  training,"  and  the 
nuclear  industry  had  "not  made  use  of  task  analyses  or 
a  comparable  formal  methodology"  for  designing  operator 
training  programs,  so  that  "decisions  tend(ed)  to  be 
made  on...  a  subjective  basis."  The  study  recommends 
the  use  of  a  systematic  task-analytic  training 
methodology,  and  a  strengthening  of  NRC  regulation  of 
training.  A  comparison  was  made  with  U.S.  Air  Froce 
pilot  training  programs  in  which  training  objectives 
and  specific  performance  criteria  are  explicitly 
spelled  out. 

(2)  A  comprehensive  risk  assessment  of  accident 
sequences  for  nuclear  power  plants  had  not  been 
performed.  Since  a  limited  amount  of  time  can  be  spent 
training  an  operator  to  respond  to  abnormal  events,  the 
events  to  be  simulated  must  be  selected  to  reflect  the 
probability  and  the  criticality  (in  terms  of  safety  and 
dollars  costs)  of  such  events.  The  final  section  of 
this  report  develops  and  demonstrates  procedures  for 
selecting  equipment  malfunctions  for  simulation,  by 
ranking  events  on  a  point  system  in  terms  of  (a) 
safety,  (b)  frequency,  (c)  potential  for  subsequent 
malfunction,  and  (d)  significant  plant  outage.  A  list 
of  malfunctions  devived  from  a  study  of  3,000  LERs 
recorded  in  a  6-  month  period  in  1978  is  appended. 


1.  Authors:  Bailey,  Jon  S. ,  &  Hughes,  Ronald  G. 

2.  Title:  Applied  Behavior  Analysis  in  Flying  Training 
Research 

3.  Source:  AFHRL-TR-7 9-3 8 ,  January  1980 

4.  Topic  Keywords:  Flight  Simulation  ; 

Behavioral  Analysis  ;  Task  Analysis  . 

5.  Short  Summary:  This  paper  lays  out  a  basic  framework  of 
a  behavioral  analysis  approach  to  simulation  training,  and 
suggests  ways  that  the  principles  of  behavior  might  be 
applied  to  simulated  flying  training. 

6.  Devices:  not  applicable 
7  .  Institutions : 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  organization:  Flying  Training  Division, 
Air  Force  Human  Resources  Laboratory,  Williams  AFB,  AZ 
85224 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  Flight  simulators  have  traditionally  been 
designed  around  "the  model  of  an  instructor  pilot  who  feels 
most  comfortable  teaching  in  an  actual  plane."  However,  the 
simulator  has  potential  for  training  which  is  not  possible 
in  the  actual  plane.  The  purpose  of  this  paper  is  to 
elucidate  scientific  principles  of  behavior  and 
hypothetically  to  apply  them  to  the  analysis  of  flying  tasks 
and  to  the  design  and  use  of  flight  training  simulators. 

The  following  basic  principles  of  behavior  are  described  and 
illustrated:  Positive  Reinforcement,  Punishment,  Shaping 

and  Chaining,  Prompting  and  Fading,  Discrimination  and 
Stimulus  Control,  and  Generalization. 

The  chief  disadvantages  of  training  in  a  real  airplane  stem 
from  the  safety  requirement:  "The  IP  (must)  put  proper 
maneuvering  above  analyzing  the  instructional  (or  learning) 
process..."  Also,  "since  the  cockpit  is  operational  and  the 
instruments  require  constant  monitoring...  the  student  may 
easily  be  overloaded  with  information  in  the  early  stages  of 
instruction  and  be  unable  to  progress...  systematically. 

No  opportunity  to  practice  a  particular  part  of  a  maneuver 
is  feasible..."  In  other  words,  the  lack  of  "realism"  in  a 
simulator  in  these  respects  constitute  a  potential  advantage 
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in  training.  However,  until  recently  these  advantages  have 
not  been  realized  because  "psychologists  specializing  in  the 
learning  process  have  not  been  involved  in  the  design  phase 
of  simulator  development." 

A  new  mode  for  simulator  design  is  proposed  which  begins 
with  a  behavioral/task  analysis.  This  analysis  can  be  done 
in  one  or  more  of  four  dimensions:  component  analysis, 
chain  or  sequence  analysis,  dimension  of  difficulty 
analysis,  and  augmented  feedback  analysis. 

The  model,  based  on  analysis  of  tasks  and  using  behavioral 
principles,  is  applied  hypothetically  to  the  example  of 
learning  how  to  land  an  aircraft.  This  example  employs  the 
principles  of  backwards  chaining,  cognitive  pretraining, 
graduated  difficulty,  and  criterion-based  individualization 
of  training. 

The  paper  concludes  with  a  recitation  of  a  number  of 
fundamental  questions,  as  yet  unanswered  by  research, 
related  to  the  training  features  and  effectiveness  of 
simulators. 
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1.  Author:  Batteiie  Human  Affairs  Research  Centers 

2.  Title:  Simulator  Fidelity  and  Training  Effectiveness: 

A  Bibliography  with  An  Annotated  Bibliography  of  Selected 
Documents 

3.  Source.:  Batteiie  Human  Affairs  Research  Centers, 
December  1982,  Safety  Technology  Program,  Project  T-2 

4.  Topic  Keywords:  Simulator  Fidelity  ; 

Training  Effectiveness  ;  Flight  Operations  Training  ; 
Nuclear  Operations  Training  ;  Maintenance  Training  . 

5.  Short  Summary:  An  annotated  bibliography  provides  35 
annotated  references  and  a  nonannotated  bibliography 
provides  216  references  (including  citations  of  the 
annotated  references)  relating  to  simulator  fidelity  and 
training  effectiveness. 

6.  Devices: 

a.  Procedural  Task  Trainers 

b.  Nuclear  Power  Plant  Operations  Simulator 

c.  Flight  Simulators 

d.  Process  Simulators 

7 .  Insti tutions : 

a.  Sponsor:  Nuclear  Regulatory  Commission 

b.  Performing  Organization:  Batteiie  Human  Affairs 
Research  Centers,  4000  N.E.  41st  Street,  Seattle,  WA 
98105 

8.  Type  of  Article:  Review  (bibliography) 

9.  Abstract:  This  bibliography  is  divided  into  two 

sections:  Section  A  contains  references  to  35  articles  and 

reports  with  annotations  constructed  in  a  uniform  outline 
format;  Section  B  cites  215  documents  including  those  cited 
in  Section  A,  without  annotation.  The  subject  matter  of 
references  in  Section  A  relates  to  "issues  of  the 
psychological  aspects  of  simulator  fidelity  and  the 
effectiveness  cf  training  simulators."  The  subject  matter  of 
references  in  Section  B  related  to  "the  role  of  simulators 
in  operator  training."  Since  the  bibliography  was  prepared 
as  part  cf  a  program  for  the  Nuclear  Regulatory  Commission, 
there  is  more  emphasis  on  issues  pertinent  to  nuclear  power 
plant  operator  training  than  would  be  found  in  a  more 
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general-interest  bibliography. 

Annotated  bibliographies  cite  author,  title,  source,  and 
date  of  the  reference,  and  are  outlined  as  follows: 

I.  General  Summary  of  Document 

.  A.  Focal  Industry 

B.  Type  of  Document 

C.  Basic  Objective(s) 

D.  Major  Findings  or  Recommendations 

II.  Simulator  Topic  Focus 

A.  Uses  of  Simulator  Addressed 

B.  Types  of  Simulators  Addressed 

III.  Simulator  Fidelity 

A.  Fidelity  Dimensions  and  Definitions 

B.  Explicit  Statement  of  Required  Fidelity  Level 

C.  Variables  Affecting  Required  Fidelity  Level 

D.  Criteria  for  Determining  Required  Fidelity 
Level 

E.  Methods  for  Determining  Required  Fidelity 
Level 

IV.  Simulators  in  Training  Systems 

A.  Relationship  to  Training  for  Specified  Tasks 

B.  Relationship  to  Training  Effectiveness 

C.  Methods  for  Establishing  Training 
Ef  f ectiveness 

D.  Simulators  as  Part  of  Training  Systems 

A  rough  breakdown  of  the  annotated  section  by  subject 
content  is  as  follows:  Flight  Operations  Training,  9 
documents  (2  are  empirical  studies);  Nuclear  Reactor 
Operations  Training,  5  documents  {none  are  empirical 
studies);  Other  Operations  Training,  5  documents  (2  are 
empirical  studies);  Maintenance  Training,  3  documents  (none 
are  empirical);  General  and  Miscellaneous,  13  documents. 


1.  Authors:  Baum,  David  R. ,  Clark,  Chriss, 

Coleman,  T.  Patrick,  Lorence,  Steve,  Persons,  Warren,  & 
Miller,  Gary 

2.  Title:  Maintenance  Training  System:  6883 
Converter/Flight  Control  Test  Station 

3.  Source:  Technical  Report,  AFHRL-TR-7 8-87 

4.  Topic  Keywords:  Simulation  ;  Maintenance  Training  ; 
Transfer  of  Training  ;  Fidelity  . 

5.  Short  Summary:  The  design  features  and  operation  of  a 
maintenance  training  system  (the  6883  Converter/Flight 
Control  Test  Station)  are  described.  The  description 
includes  system  capabilities  for  obtaining  data  on  various 
dimensions  of  device  utilization  and  training  methodology. 
The  simulator  is  discussed  both  as  a  prototype  trainer  and  a 
research  tool. 

6.  Device:  6883  Conver ter/ FI ight  Control  Test  Station,  a 
dual-  computer  interactive  training  device 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Honeywell  Systems  and 
Research  Center,  2600  Ridgway  Parkway,  Minneapolis,  MN 
55413 

8.  Type  of  Article:  Conceptual/Methodological 

9.  Abstract: 

a.  Study  Synopsis:  This  paper  describes  the  design 
features  and  operation  of  a  dual-computer  driven 
maintenance  training  system  (6883  MTS),  including 
system  architecture,  hardware  and  software,  instructor 
functions  and  procedures,  and  student  interaction.  It 
includes  a  hypothetical  troubleshooting  task  learning 
scenario  involving  the  interactions  between  in  ructor, 
student,  and  computer.  This  report  also  discuses  use 
of  the  trainer  to  explore  research  issues  in  three 
areas:  (1)  transfer  of  training,  (2)  training 

techniques,  and  (3)  automated  performance/proficiency 
measurement . 

Prior  to  development  of  the  6883  MTS,  I-level  Air  Force 
maintenance  training  had  been  conducted  on  actual 
equipment.  The  following  shortcomings  of  actual 


equipment  use  in  training  were  cited:  (1)  cost,  (2) 
inflexibility  to  train  for  part- tasks,  (3)  low 
reliability,  (4)  high  risk  of  severe  injury  to 
trainees,  (5)  high  risk  of  damage  to  equipment  through 
student  error,  (6)  feedback  requires  instructor's 
continued  presence. 

Psychological  rather  than  engineering  fidelity  was  the 
key  to  developing  a  training  system  where  the  student 
performs  (simulated)  maintenance  tasks  monitored  in  an 
automated  closed  loop. 

A  number  of  trade-offs  were  made  in  the  design  of  the 
MTS.  To  ensure  cost-effectiveness,  a  philosophy  of 
training  by  representative  example  was  adopted. 
Frequently  cost  reductions  could  be  achieved  by  using 
the  same  functionally  simulated  hardware  to  train  a 
variety  of  different  procedures.  Concentration  of 
simulated  hardware  in  a  single  location  reduced 
mechanical  assembly  costs.  For  example,  each  bay  of 
the  actual  test  station  has  an  over  temperature 
indicator  that  is  tested  during  turn-on  procedures. 
Because  this  test  procedure  is  the  same  for  all  four 
indicators,  only  one  of  the  four  indicators  is 
functionally  simulated  though  all  four  indicators  are 
represented  visually. 

b.  Authors'  Conclusions:  There  are  at  least  three 
general  areas  of  technical  training  research  that  can 
be  addressed  using  the  6883  MTS:  (1)  transfer  of 

training,  (2)  advanced  training  techniques,  and  (3) 
automated  performance/proficiency  measurements.  The 
6883  MTS  provides  a  tool  for  collecting  appropriate 
data  on  the  question  of  transfer  of  training;  measures 
obtained  on  this  system  could  help  achieve  an 
acceptable  level  of  objectivity,  reliability,  and 
validity  in  proficiency  measurement.  The  6883  MTS 
provides  a  test  bed  for  evaluating  the  training  utility 
of  freeze,  playback,  and  other  techniques.  It  also 
provides  a  research  tool  for  evaluating  the  relevance 
of  its  many  possible  recorded  performance  measures  to 
practical  proficiency  and  skill  level. 
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1.  Authors:  Baum,  David.  R. ,  Smith,  Deborah  A., 

Klein,  Gary  A.,  Hirshfelc,  Stephen  F. ,  Swezey,  Robert  W. ,  i 
Hays ,  Robert  T. 

2.  Title:  Specif ication  of  Training  Simulator  Fidelity:  A 
Research  Plan 

3.  Source:  I’.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  558, 

February  1982 

4.  Topic  Keywords:  Fidelity  ;  Fidelity  Specification  ; 
Fidelity  Guidance  ;  Maintenance  Training  ;  Simulation  . 

5.  Short  Summary:  This  study  provides  (1)  a  theoretical 
review  and  discussion  of  issues  pertaining  to  the 
relationship  between  simulator  fidelity  and  training 
effectiveness;  -,2)  a  framework  for  fidelity  research  in 
maintenance  training;  and  (3)  specific  experimental  pilot 
study  designs  to  investigate  the  impact  of  varying  levels  of 
fidelity  on  training. 

6 .  Devices :  None 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (PERI-IE),  5001 
Eisenhower  Avenue,  Alexandria,  VA  22333 

b.  Performing  Organization:  Honeywell  Systems  and 
Research  Center,  2600  Ridawav  Parkway,  Minneapolis,  MN 
5  541  3 

8.  Type  of  Article:  Methodological 

9.  Abstract :  This  interim  report  provides  the  theoretical 

basis  and  design  plans  for  empirical  research  on  the 
relationship  between  simulator  fidelity  and  training 
effectiveness.  i.The  actual  conduct  of  one  of  the  proposed 
experiments  and  its  results  are  presented  in  a  subsequent, 
final  report'.  The  paper  is  divided  into  three  principal 
sections:  1)  Introduction  and  Background;  (2)  A  Framework 

for  Fidelity  Research  in  Maintenance  Training;  and  ;  3) 

Pilot  Gtudy  Design  Considerations  and  Paradigms. 

a.  Introduction  and  Background.  A  review  of  the 
literature,  site  visits  and  interviews  with  concerned 
agencies,  and  a  fidelity  research  issues  workshop, 
yielded  the  following  list  of  issues  in  simulator 
training:  ;i)  Fidelity — its  definition,  its 

impi lcations  in  transfer  of  training,  its  dependence  on 
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actual  equipment  characteristics,  its  sensitivity  to 
the  training  environment,  and  its  place  in  the  context 
of  task  variables  such  as  task  type,  task  difficulty, 
task  criticality,  etc.;  (2)  Util ization--the 
incorporation  of  the  simulator  into  the  training 
program  and  its  acceptance  by  users;  and  (3) 
Measurement  iof  training  effectiveness — whether  by 
transf er-of- training,  by  ratings  of  experts,  or  by 
reference  to  analytic  models  of  learning. 

b.  A  Framework  for  Fidelity  Research  in  Maintenance 
Training.  The  framework  is  to  be  structured  by  the 
distinction  between  physical  and  functional  aspects  of 
fidelity  proposed  by  Hays  (1980),  which  suggests  the 
following  design: 


PHYSICAL  SIMILARITY 
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Medium 
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X 
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Three  groups  of  maintenance 

tasks 

are  identified:  (1) 

Procedural;  (2)  Perc 

eptual- 

Motor ; 

and  ( 3 )  ' 

Cognitive-- 

based  on  the  emphasis 

which 

these 

dimensions 

receive  in 

the  task.  Of  the  three,  Procedural  Maintenance  tasks 
were  eliminated  from  further  investigation  in  this 
study  because  it  was  felt  that  adequate  research  had 
already  been  done  to  show  conv incingly  that  high 
training  simulator  fidelity  was  not  a  requirement  in 
teaching  this  type  of  task. 

c.  Pilot  Study  Design  Considerations  and  Paradigms. 
General  design  considerations  were  (1)  criteria  of 
tasks  to  be  learned;  (2)  manipul ation  of  fidelity; 
ana  ( 31  transfer  of  training  paradigm. 

The  task  selected  for  investigation  of  the  perceptual- 
motor  maintenance  task  group  was  the  truing  cf  a 
Li  cycle  wheel,  a  task  easily  studied  in  a  laboratory, 
i  n  which  the  reuuii  serf  orrr.ance  was  felt  tc  be 
representative  of  Army  maintenance  tasks.  Six 
different  studies  were  proposed  for  this  task. 

The  task  proposed  for  investigation  of  the  cognitive 
maintenance  task  group;  was  the  playing  of  the  card-game 
of  bridge,  selected  because  it  is  a  decision  task  that 
car.  be  analyzed  into  elements.  A  paradigm  for  analysis 
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information  theory  and  a  completed 
rks  of  proficient  CPR  paramedics. 


1.  Authors:  Beck,  Isabel,  &  Monroe,  Bruce 

2.  Title:  Some  Dimensions  of  Simulation 


3.  Source:  INSGROUP,  Inc.,  February  1969 

4.  Topic  Keywords:  Simulation  Games  ;  Simulator  Fidelity 


5.  Short  Summary:  This  theoretical  paper  summarizes  some 
broad  aspects  of  simulation,  with  emphasis  on  simulation 
games,  in  the  context  of  education  in  general. 

6.  Devices:  not  applicable  (a  theory  paper) 

7.  Institutions: 

a.  Sponsor:  INSGROUP,  INC.  (Instructional  Systems 
Group) ,  640  Sea  Breeze  Drive,  Seal  Beach,  CA  90740 

b.  Performing  Organization:  Same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  paper  summarizes  some  broad  aspects  of 
simulation,  principally  of  simulation  games,  in  the  context 
of  education  in  general.  Definitions  and  characteristics  of 
simulation  are  given,  with  the  four  principal 
characteristics  of  simulation  identified  as  follows: 

(1)  Starts  with  an  analogous  situation 

(2)  Provides  for  low  risk  input 

(3)  Feeds  back  consequences  symbolically 

(4)  Is  replicable 

These  characteristics  are  used  to  help  make  distinctions 
between  role  playing,  games,  and  simulation.  The  dimensions 
of  simulation  discussed  are: 

(1)  The  reality/fidelity  dimension.  Two  aspects  are 
mentioned:  physical  and  time  fidelity 

(2)  The  manipulation  of  complexity  of  decisions 

(3)  Curriculum  content  aspects 

(4)  The  source  or  originator  of  the  model  on  which  the 
simulation  is  based 

(5)  Replicability  of  the  simulation  for  the  same  player 
or  for  subsequent  players 

(6)  Evaluation  or  measurement  of  performance 
Counter-based  simulation  games  are  discussed 
separately. 

The  authors  point  out  that  in  simulation  game  instruction, 
the  learning  of  process,  particularly  decision-making,  far 
out  weighs  the  learning  of  content. 
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Advantages  and  disadvantages  of  learning  through  simulation 
are  listed.  Advantages  of  simulation  over  direct  experience 
in  the  operational  environment  are:  cost,  control  over  time 
(either  to  compress  or  to  expand  from  real-world  time) ,  and 
ability  to  experiment.  Disadvantages  cited  include  design 
problems,  chiefly  in  fidelity. 
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1.  Authors:  Blaiwes,  Arthur  S. ,  &  Regan,  James  J. 

2.  Title:  An  Integrated  Approach  to  the  Study  of  Learning, 
Retention,  and  Transfer  -  a  Key  Issue  in  Training  Device 
Research  and  Development 

3.  Source:  NAVTRADEVCEN  Technical  Report  IH-178,  August 
1970 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Retention  of  Learning  . 

5.  Short  Summary:  Issues  which  play  significant  roles  in 
research  into  Learning,  Retention,  and  Transfer  are 
identified  and  applied  to  the  characterization  of  181 
reports  in  the  transfer  of  training  literature. 

6.  Devices:  not  applicable 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Device  Center,  Orlando,  FL 

b.  Performing  Organization:  Human  Factors  Laboratory, 
Naval  Training  Device  Center,  Orlando,  FL 

8.  Type  of  Article:  Review/Theoretical 

9.  Abstract:  This  paper  attempts  to  identify  distinct  and 
meaningful  issues  which  characterize  research  into  learning, 
retention,  and  transfer,  and  then  to  categorize  181  reports 
according  to  these  characteristic  issues.  Relationships 
between  learning,  retention,  and  transfer  are  briefly 
discussed,  and  three  representative  experiments  are 
summarized,  and  four  major  features  of  a  research  program 
are  presented. 

Four  general  issues  are  identified  which  characterize 
Learning,  Retention,  and  Transfer  research: 

(1)  which  subtasks,  fund  operationally,  should  be 
included  in  the  training  simulation; 

(2)  those  variations  in  stimulus  and  response 
characteristics  of  the  training  system  which  should  be 
incorporated; 

(3)  which  instructional  devices,  materials,  and  methods 
should  be  introduced  to  improve  learning  and  transfer; 

(4)  how  much  generalization  should  be  built  into 
training  devices.  " 

In  addition  to  these  issues,  the  181  reports  are 
characterized  according  to  "whether  they  pertain  to  the 
topic  of  learning,  retention  and/or  transfer;  whether  the 


tasks...  could  be  described  as  verbal,  motor,  perceptual, 
signal  monitoring,  complex,  or  procedural;  and  whether  and 
what  kind  of  other  independent  variables  were  manipulated. " 

A  summary  is  given  of  trends  in  the  literature  which  emerged 
in  the  course  of  this  analysis,  pertaining  especially  to  the 
transfer  relationship  (i.e.,  what  methods,  materials  and 
devices  enhance  transfe  ) . 

The  discussion  of  the  relationships  between  learning, 
retention  and  transfer  focusses  on  the  specificity  of  the 
transfer  issue.  Theories  are  mentioned  which  suggest  the 
importance  of  general  or  nonspecific  factors  in  learning, 
and  the  importance  of  extra-experimental  learning  in  the 
transfer  of  training  context. 

The  authors  also  identify  four  major  concerns  of  their 
research  program: 

(1)  providing  rich  and  diverse  experimental  situations 
with  which  to  examine  the  rules  of  transfer; 

(2)  evaluating  the  effects  of  the  length  of  the 
transfer  test; 

(3)  evaluating  the  effects  of  the  difficulty  of  the 
training  task; 

(4)  evaluating  the  effects  of  variations  in  the 
training  objectives. 

One  experiment  concerned  with  the  task  difficulty  issue,  and 
two  experiments  concerned  with  the  fidelity  issue,  were 
briefly  summarized. 


1.  Author:  Brock,  John  F. 


2.  Title:  Maintenance  Training  and  Simulation:  Design 
Process  and  Evaluation  Criteria 

3.  Source:  Proceedings  of  the  Human  Factors  Society  22nd 
Annual  Meeting,  Detroit,  MI,  October  16-19,  1978,  260-266 

4.  Topic  Keywords:  Maintenance  Training  Simulation  ; 
Maintenance  Troubleshooting  ;  Cost-Effectiveness  ; 
Performance  Evaluation  . 

5.  Short  Summary:  Different  types  of  maintenance  tasks  and 
methods  of  training  via  simulation  are  identified.  A  review 
of  evaluative  practices  suggests  some  changes  to  more 
accurately  reflect  the  cost  of  training  systems  to  the  user 
and  an  adaptive  model  of  instructioal  system  evaluation  is 
proposed. 

6.  Devices: 

a.  Hagan  Automatic  Boiler  Control  System 

b.  Bessler  Cue  -  See  audiovisual 

7.  Institutions: 

a.  Sponsor:  Navy  Personnel  Research  and  Development 
Center,  San  Diego,  CA  92152 

b.  Performing  Organization:  same 

8.  Type  of  article:  Conceptual 

9.  Abstract:  This  paper  endeavors  to  identify  maintenance 
training  categories  and  training  simulation  system 
characteristics,  and  to  critique  methods  of  evaluaing 
maintenance  training  systems,  and  makes  suggestions  for 
imporvements  in  system  analysis  and  evaluation. 

Three  kinds  of  maintenance  task  categories  are  identified: 

(1)  Procedural,  which  includes  Preventive  Maintenance, 
Systems  Checkout,  and  General  Inspection, 

(2)  Psychomotor,  which  includes  Calibration,  Alignment, 
and  Repair, 

(3)  Schema,  which  includes  Fault  Detection  and  Trouble¬ 
shooting. 

Although  analyses  of  procedural  tasks  have  been  heavily 
documented,  analyses  of  the  more  complex  psychomotor  tasks 
have  been  less  satisfactory.  Most  difficult  to  analyze  have 
been  schema  tasks — fault  detection  and  troubleshooting. 
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Attempts  to  link  schema  theory  with  performance  have  been 
futile.  "...The  development  of  analytic  techniques  to 
discover  the  schema  of  maintenance  technicians  is  critical 
and,  with  the  help  of  artificial  intelligence  systems,  has  a 
high  probability  of  success." 

Even  where  task  analyses  are  precise,  and  lead  to 
identifiable  learning  objectives,  the  problems  of  meeting 
learning  objectives  remain  complex  and  their  solutions  rely 
to  a  great  extent  on  intuition. 

The  advantages  of  simulation  in  maintenance  training  are 
summarized  briefly  as:  Cost,  Reliability,  Modifiability, 
Hands-on  Opportunities,  Safety,  Evaluation  of  Student 
Performance  as  Good  or  Better. 

Instructional  systems  available  to  the  maintenance  training 
designer  are  identified  as  follows:  Actual  Equipment, 
Mock-Up,  3-Dimensional-2-Dimensional  Mix,  3-D  Schematic, 
2-D/Image  Interface,  2-D/Keyboard  Interface,  and  Projected 
Displays.  The  problem  lies  not  in  available  techniques,  but 
in  "determining  what  skills  must  be  taught." 

The  measurement  of  system  effectiveness  by  tests  at  the  end 
of  the  training  period  do  not  adquately  reflect  the  true 
effectiveness  of  the  training  in  the  total  service  context; 
it  is  necessary  to  evaluate  training  by  measuring  field 
performances  of  course  graduates,  via  records  of  system 
downtimes,  mean  times  to  repair,  field  supervisor  ratings, 
and  so  on.  "Until  the  operational  community  can  provide  the 
training  community  with  evidence  that  a  particular  way  of 
training  is  improving  the  way  systems  work,  training  design 
people  will  continue  to  rely  on  schoolhouse  measures 
exclusively. " 

Likewise,  cost  analyses  which  only  measure  cost  at  the 
schoolhouse  level,  and  not  the  impact  of  training  in  the 
field,  fail  to  evaluate  the  totality  of  costs  to  the 
military  service  which  is  the  ultimate  training  system  user. 
Cost  differentials  realized  in  the  field  through  more 
efficient  performance  of  equipment  and  reduced  down-time  as 
a  result  of  training  are  just  as  important  as  Transfer 
Effectiveness  Ratios. 
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1.  Author:  Caro,  Paul  W. 

2.  Title:  Aircraft  Simulators  and  Pilot  Training 

3.  Source:  Human  Factors  ,  1973,  15(6),  502-509 

4.  Topic  Keywords:  Flight  Simulation  ; 

Functional  Context  Training  ;  Transfer  of  Training  . 

5.  Short  Summary:  A  review  of  the  role  of  simulators  in 
pilot  training  stresses  training  techniques  over  physical 
simularity  of  the  training  device  to  the  aircraft  as  the 
critical  factor  in  determining  training  effectiveness. 

6.  Devices:  GAT-2  instrument  trainer;  Cockpit  mockup; 
1-CA-l  Trainer;  2-B-24  Simulator 

7.  Institutions: 

a.  Sponsor:  Human  Resources  Research  Organization  , 

.  Fort  Rucker,  AL  36362 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Theoretical/Review 

9.  Abstract:  The  thesis  of  this  paper  is  that  "a  proper 
training  program  can  compensate  for  lack  of  physical 
similarity  between  the  training  device  and  the  aircraft,  but 
a  realistic  simulator  is  a  poor  substitute  for  competent 
training....  The  key  is  the  program,  not  the  hardware." 

In  support  of  this  contention,  specific  techniques  for 
optimizing  the  effectiveness  of  training — using  either 
simulators  or  less  sophisticated  training  devices — are 
described,  and  several  experiments  are  cited  in  which 
devices  as  crude  as  "a  cockpit  mockup  made  of  plywood  and 
photographs  by  unskilled  labor... at  a  cost  of  about  $30" 
produced  substantial  transfer  of  training.  The  techniques 
which  are  described  are  summarized  as  follows: 

a.  Functional  context  training: 

b.  Individualization  of  training; 

c.  Sequencing  of  instruction; 

d.  Minimizing  over-training; 

e.  Efficient  utilization  of  personnel 

f.  Use  of  incentive  rewards 

g.  Crew  training 

h.  Peer  training 

i.  Minimizing  equipment  costs 

j.  Objective  performance  measurement 

The  author  maintains  that  these  techniques  are  based  on  an 
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"orientation  that  training  is  a  technology  which  can  be 
engaged  in,  after  appropriate  training,  by  reasonably  bright 
and  adaptable  people,  not  an  art  which  is  an  inherent 
characteristic  of  the  'good  instructor.'" 

The  motivation  and  training  of  instructors  are  stressed. 

The  attitude  of  an  instructor  is  usually  reflected  by  the 
student;  design  and  management  of  the  training  program 
should  take  this  into  consideration  by  employing  techniques 
such  as  the  use  of  incentive  rewards  cited  above. 


1.  Author:  Caro,  Paul  W. 


2.  Title:  Some  Factors  Influencing  Air  Force  Simulator 
Training  Effectiveness 

3.  Source:  HumRRO-TR-77-2 ,  March  1977 

4.  Topic  Keywords:  Flight  simulation  ;  Platform  motion  ; 
Simulation  fidelity  ;  Training  effectiveness  . 

5.  Short  Summary:  A  study  of  ten  current  U.S.  Air  Force 
simulator  training  programs  identified  factors  influencing 
effectiveness,  but  discovered  that  no  quantitative 
evaluation  had  been  done  which  would  assess  the  relative 
impact  of  these  factors. 

6.  Devices: 

a.  FB-111  Aircraft  Simulator 

b.  C-5  and  C-5A  Simulators 

c.  F-106  Simulator 

d.  A-7D  Simulator 

e.  GAT- 2 

f.  Advanced  Simulator  for  Pilot  Training 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research 
(NL) ,  Bolling  AFB ,  DC  20332 

b.  Performing  organization:  Human  Resources  Research 
Organization,  300  North  Washington  Street,  Alexandria, 
VA  22314 

8.  Type  of  Article:  Review 

9.  Abstract:  The  study  described  in  this  report  was 
undertaken  to  identify  "the  principal  factors  that  influence 
the  effectiveness  of  Air  Force  simulator  training  programs. 
The  report  is  divided,  after  the  introduction,  into  three 
main  sections: 

a.  Methods  Used  to  Determine  Simulator  Effectiveness. 
This  section  discusses  characteristics  and  applications 
of  ten  different  investigative  models: 

(1)  Transfer  of  Training  Model 

(2)  Self-Control  Transfer  Model 
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(3)  Pre-existing  Control  Model 

(4)  Uncontrolled  Transfer  Model 

(5)  Simulator-to-Simulator  Transfer  Model 

(6)  Backward  Transfer  Model 

(7)  Simulator  Performance  Improvement  Model 

(8)  Simulator  Fidelity  Model 

(9)  Simulator  Training  Program  Analysis  Model 

(10)  Opinion  Survey  Model 

b.  Principal  Factors  Influencing  Simulator  Training 
Effectiveness.  This  section  discusses  the  following 
factors  derived  both  from  the  survey  and  from  a 
literature  search: 

(1)  Simulator  Design 

(2)  Design  for  Training 

(3)  Visual  Fidelity 

(4)  Motion  Fidelity 

(5)  Handling  Characteristics 

(6)  Training  Programs 

(7)  Personnel 

(8)  Instructors 

(9)  Attitudes 

(10)  Expectations 

(11)  Miscellaneous 

c.  Recommendations.  Recommendations  address  most  of 
the  factors  listed  above. 

Regarding  methods  used  to  determine  simulator 
effectiveness,  "a  general  absence  of  formal  (i.e., 
objective,  quantitative)  evidence  of  the  value  of  the 
simulator  training  activities  surveyed"  was  found, 
although  most  personnel  surveyed  believed  strongly  in 
the  cost-effectiveness  of  their  simulation  training 
programs.  Of  the  ten  programs  surveyed,  none  had  used 
the  Transfer  of  Training  Model  to  evaluate  program 
effectiveness.  The  author  comments  that  "it  may  be 
that  simulator  training  has  become  too  well  accepted  by 
the  Air  Force. " 

In  the  discussion  of  principal  factors  influencing 
training  ef f ectivenss,  a  great  deal  of  attention  is 
placed  on  device  design  fidelity.  It  is  noted  that 
inclusion  of  instructional  features  such  as  freeze, 
playback,  etc.,  which  are  believed  to  enhance  the 
teaching  function,  lowers  the  fidelity  of  the  training 
equipment.  The  capability  of  the  device  to  train 
legitimately  out-weighs  considerations  of  fidelity  per 
se.  With  this  in  mind,  it  is  suggested,  for  example, 
in  connection  with  visual  fidelity,  that  "it  sometimes 
appears  that  unnecessary  emphasis  is  placed  upon 
designing  visual  displays  that  look  like  the  "real 
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world"  viewed  through  the  aircraft's  wind  screen.... 

It  is  likely  that  the  display  design  requirements  are 
more  responsive  to  fidelity  goals  than  to  training 
necessities."  A  practical  obstacle  to  sacrificing 
fidelity  in  the  interest  of  cost-effectiveness  is  the 
opposition  of  instructors  who  insist  on  the  importance 
of  realism  of  aircraft  "feel"  in  the  simulator,  even 
when,  as  in  the  case  of  simulated  motion,  the 
contribution  of  the  simulation  to  training  appears  weak 
when  measured  experimentally. 

Recommendations  focus  on  the  need  for  emphasizing 
validation  of  simulator  training  programs  objectively, 
and  researching  simulator  design  characteristics  in 
relation  to  training  objectives  in  a  systematic  way. 
"Design  concepts  should  be  sought  that  will  emphasize 
the  simulator's  training  purposes  rather  than  the 
characteristics  of  the  aircraft  being  simulated." 

Overall,  it  was  noted  that  both  literature  and 
personnel  surveyed  failed  to  agree  on  simulator  design 
specifics  in  many  areas.  "An  obvious  conclusion  to  be 
drawn  from  the  present  study  is  that  too  little 
attention  has  been  paid  to  date  to  the  effectiveness  of 
simulator  training. " 


1.  Authors:  Eddowes,  Edward  E. ,  &  Waag,  Wayne  L. 

2.  Title:  The  Use  of  Simulators  for  Training  In-Flight  and 
Emergency  Procedures 

3.  Source:  NATO  Advisory  Group  for  Aerospace  Research  and 
Development  AGARDograph  No.  248,  June  1980 

4.  Topic  Keywords:  Flight  Simulation  ; 

Emergency  Flight  Procedures  ;  Measurement  Content  Validity 
;  Measurement  Empirical  Validity  ;  Performance  Measurement 

• 

5.  Short  Summary:  This  report  examines  the  role  of  flight 
simulators  in  training  with  a  focus  on  a  few  key  issues, 
reviews  a  number  of  empirical  studies,  and  discusses  in 
detail  the  problem  of  performance  measurement. 

6.  Devices: 

a.  Advanced  Simulator  for  Pilot  Training  (ASPT) 

b.  C-5A  Operational  Flight  Simulator ■ (OFS) 

c.  Ch-47  Operational  Flight  Simulator 

d.  Device  2F87f  Operational  Flight  Trainer 

e.  SNJ  Link  Trainer 

f.  l-CA-2  SNJ  Link  Trainer 

7 .  Institutions : 

a.  Sponsor: 

NATO  Advisory  Group  for  Aerospace  Research  and 
Development,  7  Rue  Ancelle,  9220  Neuilly-sur-Seine, 
France 

b.  Performing  Organization:  Williams  AFB ,  AZ  85224, 

Air  Force  Human  Resources  Laboratory 

> 

8.  Type  of  Article:  Review 

9.  Abstract:  This  report  examines  the  role  of  flight 
simulators  in  training  with  a  focus  on  the  following  key 

ssues:  the  essence  of  the  pilot's  flying  skill;  the 

practical  problem  of  simulator  fidelity;  empirical  evidence 
of  simulator  training  effectiveness  as  a  function  of 
simulated  visual  capability  and  motion  capability;  how 
training  strategies  reflect  the  view  of  the  simulator;  and 
the  problem  of  performance  measurement. 
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with  tracking  performance.  An  experiment  is  cited  which 
exemplified  the  latter  drawback. 

Several  experiments  are  adduced  which  support  the  view  that 
the  size  of  platform  motion  effects  on  pilot  performance  are 
so  small  in  comparison  with  other  variables  in  the  training 
environment--especiaily  subject  differences  and  trial 
effects--as  to  be  negligible  in  terms  of  practical 
significance,  even  though  the  statistical  significance  may 
be  high. 

Airborne  studies  have  revealed  that  visual  sensing  can  be  of 
finer  resolution  than  vestibular  or  tactile  sensing. 
Especially  for  experienced  pilots,  the  case  for  motion  is 
extremely  weak.  "At  the  Flying  Training  Division  of  the  Air 
Force  Human  Resources  Laboratory,  almost  all  possible 
maneuvers  and  tasks  for  Instrument,  Navigation,  Formation, 
Contact,  Air-to-Air,  and  Air- to-Surf ace  flight  have  been 
examined,  and  as  yet  not  a  single  one  has  been  found  that  is 
not  trainable  without  a  platform  motion  system." 

Three  transf er-of- training  experiments  are  discussed,  in 
which  motion  effects  were  eliminated,  and  the  simulators  had 
low  physical  and  functional  fidelity  in  several  other 
respects.  In  these  cases,  transfer  to  aircraft  was 
excellent,  suggesting  it  was  the  cognitive  components  that 
mattered  most;  "teaching  'judgement',  or  more  precisely  the 
building  of  a  strategy  of  flying  based  on  experience,  is  the 
critical  problem." 

The  authors  conclude  that  the  logical  place  to  teach  proper 
use  of  motion  cues  is  in  the  aircraft  rather  than  in  the 
simulator,  and  that  at  the  state  of  technology  current  as  of 
this  report,  simulators  could  safely  be  procured  without  a 
platform  motion  system;  and  that  simulators  should  be 
purchased  with  the  largest  field  of  view  consistent  with 
mission  requirements. 
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1.  Author:  Cyrus,  Michael  L. 

2.  Title:  Motion  Systems  Role  in  Flight  Simulators  for 
Flying  Training 

3.  Source:  AFHRL-TR-7 8-3  9,  August  1978 

4.  Topic  Keywords:  Flight  Simulation  ; 

Platform  Motion  Systems  ;  Transfer  of  Training  ; 

Training  Effectiveness  . 

5.  Short  Summary:  A  review  of  the  literature  pertaining  to 
the  effectiveness  of  platform  motion  in  simulator  training 
concludes  that  platform  motion  is  not  required  for  the 
learning  of  most  piloting  tasks. 

6.  Devices: 

a.  Advanced  Simulator  for  Pilot  Training 

b.  LANGLEY  Visual-Motion  Simulator 

c.  Formation  Flight  Trainer 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Flying  Training  Division, 
Air  Force  Human  Resources  Laboratory,  Williams  AFB,  AZ 
85224 

8.  Type  of  Article:  Review 

9.  Abstract:  The  use  of  expensive  platform  motion  in 
simulators  for  flying  training  has  come  under  heavy 
criticism  in  recent  years.  This  paper  reviews  literature 
relevant  to  the  role  of  platform  motion  systems  in  flight 
training  simulators  in  three  areas:  ground,  airborne,  and 
transfer  studies. 

The  largest  volume  of  research  on  motion  is  in  single-  axis 
compensatory  tracking  in  ground-based  trainers,  where  the 
subject  attempts  to  minimize  an  error  initiated  by  the 
experimenter,  by  the  instructor,  or  automatically  by  the 
device.  Two  major  drawbacks  to  platform  motion  cueing  in 
this  environment  are  identified:  (1)  its  applicability  is 
limited  to  dealing  with  control  inputs  of  a  random,  random¬ 
appearing,  or  transient  nature;  (2)  control  system  dynamics 
studied  are  often  very  dissimilar  to  aircraft  dynamics, 
sometimes  leading  to  the  interference  of  simulator  motion 
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A  study  of  the  pertinent  maintenance  tasks,  the  equipment 
being  maintained,  and  the  courses  of  instruction,  was 
conducted  via  a  review  of  literature,  on-site  visits  and 
interviews,  mailed  questionnaires,  a  study  of  Maintenance 
Data  Collection  Records,  and  a  review  of  two 
apprentice-level  training  courses. 

The  first  result  of  the  study  was  a  model  of  a  "Generalized 
Trainer  System"  presented  in  the  form  of  block  diagrams; 
shown  in  this  report  is  the  "Top  Level",  or  overall  system 
diagram.  The  task  analysis  yielded  data  on  task  frequency 
and  difficulty.  The  training  analysis  yielded  a 
six-category  classification  of  course  content.  A  number  of 
alternatives  to  the  present  training  programs  are  briefly 
described. 

The  concept  of  developing  separate  devices  for  training 
technical  school  students  and  field  maintenance  personnel 
rests  on  two  principles: 

(1)  Basic  support  skills  involve  psychomotor  skills 
best  learned  by  "hands-on"  training  with  devices  of 
high  physical  fidelity. 

(2)  Troubleshooting  is  a  form  of  problem  solving  skill 
which  does  not  require  high  fidelity  simulation. 

Based  on  these  principles,  the  design  models  for  two 
devices,  a  Basic  Skills  Trainer,  and  a  Specific  Skills 
Troubleshooting  Trainer,  were  described.  A  third  concept, 
that  of  a  portable  "System-Specific  Repair  Procedures 
Trainer"  is  summarized,  but  no  functional  specification  was 
prepared. 

The  authors  recommend  that  the  specified  trainers  be  built 
in  prototype  and  used  for  training  and  research  purposes. 
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1.  Authors:  Condon,  Charles  F.  M. ,  Ames,  Lawrence  L. , 
Hennessy,  John  R. ,  &  Shriver,  Edgar  L. 

2.  Title:  Flight  Simulator  Maintenance  Training; 

Potential  Use  of  State-of-the-Art  Simulation  Techniques 

3.  Source:  AFHRL-TR-79-19,  June  1979 

4.  Topic  Keywords:  Flight  Simulators  ; 

Maintenance  Simulation  ;  Task  Analysis  . 

5.  Short  Summary:  A  study  of  the  need  for  simulated 
training  for  technicians  involved  in  mantaining  flight 
simulators  produces  models  for  three  prototype  maintenance 
training  devices. 

6.  Devices:  None  specifically  discussed 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Kinton,  Inc.,  1500  North 
Beauregard  St.,  Suite  205,  Alexandria,  VA  22311 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  Training  for  maintenance  of  military  flight 
simulators  is  hampered  by  a  lack  of  availability  of  these 
devices  due  to  operational  field  requirements.  The 
objectives  of  this  study  were  to  analyze  the  responsible 
maintenance  specialties'  tasks  and  their  associated  training 
programs,  to  report  potential  cost-effective  applications  of 
simulation  technology  to  impart  the  necessary  maintenance 
knowledges  and  skills,  and  to  provide  sets  of  functional 
specifications  for  maintenance  simulators. 

One  problem  associated  with  developing  simulation 
specifications  is  the  lack  of  knowledge  of  fidelity 
requirements.  The  authors  introduce  the  concept  of 
behavioral  fidelity,  which  "focusses  on  the  replication  of 
man-machine  interactions  (behaviors)  determined  as  a  result 
of  task-analytic  procedures."  It  is  observed  that  "many 
current  approaches  to. .. simulation  fidelity  were  developed 
with  respect  to  the  training  of  coordinate  psychomotor 
skills... and  are  insufficient  for  the  development  of  mental 
concepts,  problem  solving  ability,  and  strategy 
development....  Fidelity  of  simulation  should  not  be 
considered  as  a  unitary  parameter....  Conceptual  skills 
seem  to  be  especially  suited  to  low  fidelity,  low  cost 
trainers. " 
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simulator  is  needed,  and  if  so,  what  characteristics  are 
required.  Products  of  a  front-end  analysis  include:  (1)  a 
listing  of  task/skill/knowledges  to  be  trainied;  (2)  an 
established  set  of  training  priorities;  and  (3)  documents 
that  specify  the  functional  requirements  of  the  related 
hardware,  software,  and  courseware. 

Questions  addressed  in  the  development  of  the  simulator  are: 
(1)  what  is  the  intented  scope  of  the  training  system;  (2) 
what  is  the  entering  skill/knowledge  level  of  the  trainee 
population;  (3)  what  are  the  deficiencies  in  existing 
training;  and  (4)  what  is  the  set  of  on-the-job  procedures 
that  an  individual  of  the  designated  exit-level  skill  is 
expected  to  execute? 

The  following  prescriptions  govern  the  development  of 
cost-effective  simulation: 

a.  Design  to  meet,  not  to  exceed,  required 
capabilities;  do  not  overdesign  either  hardware  or 
software.  "Carefully  prepared  functional 
specifications  can  often  lead  to  dramatic  reductions  in 
the  required  fidelity  of  simulated  controls  and 
displays  without  negatively  impacting  training 
effectiveness."  Savings  are  often  realized  in  software 
by  substituting  procedural  for  modelled  software. 

b.  Design  fo  User  Acceptance;  since  users  resist  low 
fidelity  in  simulation,  regardless  of  its  objective 
merits,  it  is  practical  to  design  in  some  higher 
fidelity  features  where  the  cost  is  not  excessive. 

c.  Address  current  training  deficiencies. 

d.  Provide  added  instructional  capability,  to  free  the 
instructor  from  routine  tasks,  to  prevent  practice  of 
incorrect  responses,  and  where  appropriate,  to  permit 
self-paced  learning,  and  to  obtain  other  benefits. 

e.  Provide  appropriate  performance  measurements. 

f.  Design  for  suppor tabil ity — that  is,  cost-effective 
maintenance,  expandability/flexibility,  and 
configuration  management. 

It  is  noted  that  research  is  required  to  measure  transfer  of 
training  more  accurately,  realiably,  and  meaningfully. 
Studies  done  to  date  have  often  failed  to  do  all  these 
together. 


&  Gardner,  J. 


1.  Authors:  Clark,  C. , 

2.  Title:  Designing  Simulators  for  Practicality 

3.  Source:  Proceedings  of  1977  NTEC  Industry  Conference, 
November  15-17,  1977.  p.  319;  Technical  Report: 
NAVTRAEQUIPCEN  IH  294 

4.  Topic  Keywords:  Maintenance  Simulator  Training  ; 
Simulation  Fidelity  ;  Maintenance  Simulation  . 

5.  Short  Summary:  Factors  bearing  on  the  practicality  of 
simulation  training,  in  terms  of  cost-effectiveness,  are 
discussed  with  an  emphasis  on  resource  conservation.  Design 
issues  are  considered  in  terms  of  front-end  analysis  and  a 
case  study  of  the  6883  Maintenance  Training  System  is 
presented. 

6.  Devices:  6883  Simulator  System 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization:  Honeywell  Systems  and 
Research  Center,  Minneapolis,  MN  55440 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  paper  discusses  factors  bearing  on  the 
practicality,  in  terms  of  cost-effectiveness,  of  using 
simulators  in  place  of  actual  equipment  for  maintenance 
training. 

Simulating  maintenance  equipment  offers  potential  cost 
savings  both  in  procurement  and  in  maintenance  of  the 
training  equipment.  However,  it  is  also  important  to 
recognize  that  in  a  training  system  as  a  whole,  personnel 
costs  usually  account  for  the  major  portion  of  the  total 
cost,  and  simulators  have  even  greater  potential  to  minimize 
personnel  costs  associated  with  their  use.  Simulator-based 
training  systems  can:  (1)  improve  the  instructor/student 
ration,  cutting  instructor  costs;  (2)  reduce  student 
on-board  time;  (3)  reduce  required  maintenance  force  for 
training  equipment;  (4)  minimize  personnel  required  to 
incorporate  curriculum  and/or  engineering  changes;  and  (5) 
reduce  required  number  of  field  maintenance  personnel, 
thereby  reducing  the  required  training  throughput. 

Simulation  should  be  implemented  selectively.  A  front-end 
analysis  is  the  crucial  first  step  to  determining  whether  a 
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(3)  "Cost  Analysis",  which  will  take  into 
consideration  facilities,  equipment,  instructional 
material  and  training  of  instructors,  instructor 
time,  student  time  and  expenses,  and 
"miscellaneous " . 

At  a  general  level,  the  experimental  design  for  factors 

(1)  and  (2)  approximates  a  "ramdomized  controlled  field 
trial"  approach.  Eight  basic  treatment  groups  were 
defined  as  follows: 

(1)  Simulator-trained,  simulator-tested,  field- 
assigned  to  6883; 

(2)  Simulator-trained,  simulator-tested,  field- 
assigned  to  other  equipment; 

(3)  Simulator-trained,  actual-equipment-tested, 
field-assigned  to  6883; 

(4)  Simulator-trained,  actual  equipment-tested, 
field-assigned  to  other  equipment; 

(5)  Actual  equipment  trained,  simulator-tested, 
field  assigned  to  6883; 

(6)  Actual  equipment  trained,  simulator-tested, 
field  assigned  to  other  equipment; 

(7)  Actual  equipment  trained,  Actual  equipment 
tested,  field-assigned  to  6883; 

(8)  Actual  equipment  trained,  actual  equipment 
tested,  field-assigned  to  other  equipment. 

It  was  planned  to  select  groups  randomly  from  a  total 
of  approximately  180  F-lll  avionics  maintenance 
trainees  during  a  one-year  period. 

b.  Results:  not  applicable 

c.  Authqr's  Conclusion:  not  applicable 


(5)  Instructor  rating  of  "activity"  or  "passivity" 
of  trainee  in  2-person  team  test 

(6)  Scores  of  Estimated  Job  Proficiency  Test 
(pencil  and  paper) 

(7)  Subjective  ratings  of  field  performance  by 
supervisors 

f.  Experimental  setting/training  context: 
institutional,  hands-on 

g.  Statistical  methods:  not  specified 

h.  Variables  being  manipulated: 

(1)  Training  devices:  as  above  section  6 

(2)  Fidelity  Levels  (of  simulator) :  not  specified 

(3)  Type  of  task/skill  required:  maintenance, 
cognitive,  psychomotor,  perceptual,  procedural 

(4)  Task  Difficulty:  Medium-high 

i.  Stage  of  training:  not  specified 

j.  Trainee  sophistication:  not  specified 

k.  Incorporation  of  device  into  P.O.I.:  presumably 
lock-step 

l.  User  acceptance  or  attitude:  not  discussed 

m.  Use  of  Instructional  features: 

(1)  Intensity:  assumed  will  be  intensive 

(2)  Features  used:  not  specified 

Abstract : 

a.  Study  Synopsis:  This  report  describes  a  plan  for 
evaluating  the  cost-effectiveness  of  a  maintenance 
training  simulator  in  comparison  with  an  actual 
equipment  trainer,  on  the  basis  of  three  factors: 

(1)  "Simulator  Related  Issues",  referring  to  how 
well  simulator-trained  students  acquired  skills 
during  schooling,  as  measured  by  performance  tests 
(written  exams  and  troubleshooting  trials) . 

(2)  "Assessment  of  Job  Proficiency",  referring  to 
how  well  simulator- trained  students  perform  in  the 
field  after  graduation,  as  measured  by  scores  on 
an  Estimated  Job  Proficiency  Test,  and  subjective 
ratings  of  performance  by  supervisors  in  the  field 
at  intervals  of  3,  6,  and  9  months  after 
schooling. 


1.  Author:  Cicchinelli,  Louis  F. 

2.  Title:  Avionics  Maintenance  Training:  Relative 
Effectiveness  of  6883  Simulator  and  Actual  Equipment.  Test 
and  Evaluation  Plan 

3.  Source:  AFHRL-TR-7 9-13 ,  October  1979 

4.  Topic  Keywords:  Maintenance  Training  . 

5.  Short  Summary:  This  report  describes  an  evaluation  plan 
which  presents  eight  experimental  hypotheses  and  an 
accompanying  methodology.  A  cost-analysis  outline  bearing 
on  the  cost-effectiveness  of  a  simulator  versus  acutal 
equipment  for  instruction  of  F-lll  maintenance  trainees  is 
also  presented. 

6.  Devices: 

a.  6883  operational  test  station  equipment 

b.  6883  simulator 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization: 

Denver  Research  Institute,  University  of  Denver, 

Denver,  CO  80208 

8.  Type  of  Article:  Methodological  (an  evaluation  plan) 

a.  Number  of  groups:  8  (projected) 

b.  Description  of  Groups: 

(l)-(4)  Approximately  20  avionics  maintenance 
trainees  each  group 

(5) -(8)  Controls:  approximately  20  same,  each 
group 

c.  Tests  or  trials/timing:  7  post-tests  (includes  3 
Field  Performance  Inventory) 

d.  Number  of  different  types  of  measures:  7 

e.  Description  of  measurements  and  ratings: 

(1)  End-of- training-block  exam  scores 

(2)  Accuracy  of  trouble-shooting  solution 

(3)  Nature  of  errors  in  trouble-shooting  task 

(4)  Time  to  completion,  trouble-shooting  task 


Maneuver  motion  is  defined  (per  Gundry)  as  "that  motion 
which  arises  within  the  control  loop  and  results  from  pilot- 

initiated  changes . "  Disturbance  motion,  on  the  other 

hand,  "arises  outside  the  control  loop  and  results  from 
turbulence  or  from  failure  of  some  component  of  the 
airframe,  equipment,  or  engines...."  Disturbance  motion 
permits  more  rapid  and  accurate  control  under  simulated 
flight  conditions  in  which  such  motion  is  appropriate. 
Maneuver  motion  is  a  useful  control  cue  only  when  the 
simulated  aircraft  is  unstable,  or  when  an  usually  stable 
aircraft  is  in  a  particular  unstable  condition. 

With  this  perspective,  the  author  discusses  maneuver  motion 
relative  to  VTOL  aircraft,  concluding  that  maneuver  motion 
is  important  for  training-effective  simulation  of  certain 
unstable  conditions  such  as  the  hover  mode;  however, 
maneuver  motion  has  not  been  shown  to  contribute  to  transfer 
of  training  in  stable,  easy-to-fly  aircraft.  "The  magnitude 
of  motion  in  a  simulator  is  probably  of  less  importance  than 
the  promptness  and  cor  relatedness  of  such  motion." 

With  regard  to  disturbance  motion,  further  differentiation 
is  made  between  uncorrelated  disturbance  motion — irregular, 
random,  or  relatively  constant  motion  such  as  turbulence  or 
engine  vibration — and  correlated  motion,  which  is  a 
consequence  of  events  that  are  of  immediate  interest  to  the 
pilot,  such  as  equipment  failure  or  imbalancing  due  to 
jettison,  etc.  As  with  maneuver  motion  cues,  the 
cor  relatedness  of  the  cues  is  much  more  important  than 
magnitude,  unless  the  magnitude  of  excursion  is  a 
significant  cue  in  itself. 

It  is  recommended  that  correlated  disturbance  motion  cues 
resulting  from  aircraft  failures  and  malfunctions  and  events 
such  as  weapons  firing,  etc.,  be  provided  in  a  training 
simulator;  and  that  inclusion  of  some  uncorrelated  motion 
cues  for  manipulating  training  difficulty  might  also  be 
useful . 

Empiracal  research  needs  to  be  conducted  to  investigate 
these  hitherto  neglected  aspects  of  motion  cuing.  Although, 
if  the  simulated  aircraft  is  relatively  stable,  or  if  the 
maneuvers  being  trained  are  inherently  stable,  maneuver 
motion  cues  would  not  normally  be  required,  it  does  not 
follow  that  simulated  motion  cannot  contribute  to 
transferable  training  for  unstable  flight  maneuvers,  or  for 
conditions  where  disturbance  motion  becomes  an  important 
factor . 
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1.  Author:  Caro,  Paul  W. 

2.  Title:  The  Relationship  between  Flight  Simulator  Motion 
and  Training  Requirements 

3.  Source:  Human  Factors  ,  1979,  21(4),  498-501 

4.  Topic  Keywords:  Flight  simulation  ;  Platform  motion  ; 
Transfer  of  training  . 

5.  Short  Summary:  This  review  of  studies  made  of  simulator 
motion  in  flight  training  distinguishes  between  different 
types  of  motion  cues.  It  argues  that  the  investigation  of 
the  contribution  of  important  types  of  motion  in  training 
has  been  neglected  in  empirical  studies  to  date. 

6.  Devices:  not  applicable 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research 
(NL) ,  Bolling  AFB ,  DC  20332 

b.  Performing  organization:  Seville  Research  Corp. , 
Pensacola,  FL  32505 

8.  Type  of  Article:  Review/Theoretical 

9.  Abstract:  This  paper  re-evaluates  the  issue  of  platform 

motion  in  simulation  training,  by  examining  the  relationship 
between  specific  types  of  motion  cues  and  specific  motion- 
related  training  ojectives.  In  line  with  the  distinctions 
made  by  Gundry  (1976;  1977)  between  maneuver  motion  and 

disturbance  motion,  the  author  argues  that  the  many  studies 
which  have  failed  to  demonstrate  a  significant  transfer 
effect  from  simulated  motion  in  training,  may  have  failed 
because  they  emphasize  maneuver  motion  cues  in  stable 
aircraft  whose  contribution  to  performance  is  admittedly 

si ight. 

A  number  of  studies  are  cited  which  support  the  view  that 
(a)  while  simulated  motion  does  affect  pilot  performance  in 
the  simulator,  and  strong  correlations  have  been  found 
between  performance  in  the  aircraft  and  performance  in  the 
simulator  due  to  motion,  nevertheless  (b)  simulated  motion 
in  training  does  not  benefit  transfer  performance  in  the 
aircraft,  or  has  at  best  only  very  marginal  effect. 

The  author  explained  this  paradox  with  the  argument  that 
"regardless  of  other  merits,  all  such  studies  have  been 
deficient  in  that  they  have  not  paid  sufficient  attention  to 
the  function  and  composition  of  the  motion  being  provided." 
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learning  proce'- ^  has  never  been  developed  for  simulator 
training. " 

Two  major  problems  in  simulator  testing  are:  (a)  simulator 
tests  ignore  training  suitability;  and  (b)  poor  feedback  to 
the  simulator  designer.  Evaluation  of  simulators  should  be 
focussed  primarily  on  how  well  they  train,  and  only 
secondarily  on  hardware  and  performance  specifications, 
reliability  and  maintainability.  "Some  of  the  simulators 
that  have  been  accepted  in  the  past  were  so  poorly  suited  to 
training  needs  that  they  literally  could  not  be  used. "  With 
respect  to  feedback  to  the  designer,  "not  only  is  there  a 
need  to  keep  the  user  in  the  design  process,  there  is  a  need 
to  keep  the  designer  aware  of  the  user's  activities  with  the 
device  after  it  has  been  put  to  use." 

Four  problems  are  discussed  under  the  heading  of  Simulator 
Use:  (a)  Inattention  to  Techniques  of  Simulator  Training, 

which  is  associated  with  the  lack  of  a  good  model  of  the 
simulator  instructional  process.  (b)  Inadequate  Training 
for  Simulator  Instructors:  instructors  are  often  unfamiliar 
with  instructional  techniques  and/or  how  to  apply  them  to 
instructional  features  of  the  simulator.  (c)  Emphasis  Upon 
Rate  of  Simulator  Utilization,  usually  in  the  form  of 
quotas;  attempts  to  fill  up  simulator  time  with  redundant 
training  is  not  only  wasteful  of  personnel  time,  but  also 
generates  negative  attitudes  toward  the  simulator  among 
instructors  and  trainees.  (d)  Inadequate  Simulator  Cost 
Effectiveness  Data  is  a  managerial  problem  resulting  from 
incomplete  cost-  accounting  procedures  and  the  lack  of  an 
adequate  military  cost  model. 

In  conclusion,  "simulator  training  is  itself  part  of  a  much 
larger  training  system  that  includes  training  in  operational 
vehicles  and  classrooms  and  training  with  supporting 
resources....  As  long  as  we  treat  these  and  other  system 
elements  as  independent,  with  responsibilities  fractionated, 
simulator  training  will  continue  to  be  beset  with  problems 
of  importance  equal  to  or  greater  than  those  identified 
here.  " 
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5.  Short  Summary:  A  number  of  conceptual  and  managerial 
problems  in  simulator  design,  testing,  and  use  are 
discussed.  One  of  the  most  serious  problems  is  that,  to  a 
great  extent,  simulators  are  designed  to  simulate  rather 
than  to  train. 

6.  Devices:  not  applicable;  no  specific  devices  are 
mentioned 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research 

W)'  (NL) ,  Bolling  AFB ,  DC  20332 

b.  Performing  organization: 

Human  Resources  Research  Organization  ,  300  North 
Washington  Street,  Alexandria,  VA  22314 

8.  Type  of  Article:  Conceptual 

9.  Abstract:  While  technical  problems  in  development  of 
simulation  equipment  and  associated  training  programs  have 
received  considerable  attention,  conceptional  and  managerial 
problems  are  the  subject  matter  of  this  paper.  The  problems 
described  have  been  noted  by  the  author  as  common  to  all 
four  military  services.  They  are  divided  among  three  major 
phases  of  the  simulator  life  cycle: 

(1)  Design  Phase 

(2)  Testing  Phase 

(3)  Use  Phase 

Two  major  problems  in  the  Design  Phase  are:  (a)  Isolation 
of  the  Simulator  User;  and  (b)  Inattention  to  behavioral 
and  Training  Models.  Designers  are  specialists  who  are 
often  out  of  touch  with  the  user  who  originally  requested 
the  equipment.  "In  the  absence  of  a  strong  user  influence 
during  the  simulator  design  process,  decisions  can  be  made 
that  compromise  the  future  device's  training  potential." 
Behavioral  and  training  models  are  neglected  to  the  extent 
that  simulators  are  designed  to  simulate  rather  than  to 
train.  "Unfortunately,  a  well  defined  model  for  the 
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The  pilot's  flying  skill  is  seen  as  essentially  cognitive  in 
nature;  his  airmanship  lies  in  his  ability  to  "develop 
mission  plans  and  to  use  them...  Essential  pilot  skills... 
represent  primarily  the  operation  of  his  repertory  of 
cognitive  processes."  The  importance  of  this  perspective 
appears  in  the  discussion  of  the  use  of  flight  simulators 
for  emergency  procedures  training,  wherein  a  new  approach 
called  Situational  Emergency  Training  is  shown  to  focus  on 
training  good  judgement. 

The  practical  problem  of  simulator  fidelity  centers  on  the 
diminishing  ratio  of  effectiveness  to  cost  as  realism  is 
added.  Even  if  the  effectiveness  of  a  high-fidelity 
simulator  were  especially  high  with  regard  to  training  any 
one  student  on  any  one  maneuver,  budgetary  and  time 
constraints  will  limit  the  number  of  simulators  available 
and  the  number  of  pertinent  maneuvers  that  can  be  practiced 
in  a  training  session,  and  thus  reduce  the  effective 
throughput  of  students  per  dollar  and  per  hour.  The  "search 
for  higher  and  higher  realism  can  be  self-defeating,”  and 
thus  the  development  of  simulators  must  use  a  different 
criterion.  The  tr ansf er-of-t raining  test  is  recommended  as 
the  preferred  criterion. 

A  number  of  empirical  studies  of  effectiveness  using  the 
transf er-of- training  model  to  investigate  visual  and  motion 
simulation  are  summarized,  with  the  general  conclusions  that 
visual  systems  are  effective  and  platform  motion  systems 
have  dubious  value.  A  general  critique  of  this  body  of 
research  as  a  whole  finds  the  following  shortcomings: 

(1)  Differences  in  research  objectives  between 
experiments, 

(2)  Differences  in  experimental  design  and  control 
between  experiments, 

(3)  Reliance  on  subjective  judgements  of  flight 
instructors  as  a  proficiency  measure, 

(4)  A  trend  toward  small  sample  sizes,  with  a 
consequent  reduction  in  analytic  power, 

(5)  Differences  in  task  selection  between  experiments, 

(6)  Lack  of  generalizability  of  findings,  since  each 
study  tends  to  focus  on  a  specific  problem, 

(7)  Lack  of  systematic  investigation  of  the 
relationship  between  fidelity  and  effectiveness. 
"Because  of  our  current  inability  to  match  training  and 
fidelity  requirements,  it  is  likely  that  simulators 
will  continue  to  be  procured  under  the  design  goal  of 
maximum  fidelity." 

Training  strategies  reflect  the  view  of  the  simulator  either 
as  an  aircraft  substitute  or  as  an  integrated  training 
medium.  The  latter  view  fosters  a  program  which  uses  a 
variety  of  media  in  a  sequential  progression;  it  is 


exemplified  by  the  American  Airlines'  Transition  and 
Recurrency  programs  for  Captains  and  First  Officers.  The 
former,  non-integratea  approach  is  often  flawed  by  the 
obvious  discrepancies  between  simulator  and  aircraft.  The 
"Integrated  Approach"  discussed  in  this  section  parallels 
the  "Least  Cost  Sequenced  Multi-Media  Training"  described 
earlier  in  the  report. 

The  problem  of  performance  measurement  is  discussed  in 
detail,  with  special  attention  to  the  issue  of  validity 
(reliability  and  sensitivity  are  also  discussed). 
Consideration  is  given  to  two  ongoing  performance 
measurement  development  projects  in  the  U.S.  Air  Force. 
Validity  is  separated  into  the  dimensions  of  content 
validity — i.e.,  how  the  contents  of  what  is  measured  match 
meaningful  components  of  the  flight  task — and  empirical 
validity,  i.e.,  how  well  the  measurement  discriminates 
between  pilots,  correlates  with  concurrent  measures,  is 
sensitive  to  effects  of  training,  and  is  sensitive  to 
decrements  resulting  from  environmental  or  pilot  stress 
factors. 

Projections  of  future  developments  in  simulator  training 
prograns  to  include  air  combat  training,  and  some  brief 
general  conclusions  round  out  the  report. 
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5.  Short  Summary:  A  review  is  made  of  issues  and  problems 
in  maintenance  simulator  design  and  use,  and  recommendations 
are  made  for  future  research. 

6.  Devices: 

a.  ECII/EC-3  Maintenance  simulator 

b.  E-4  Fire  control  troubleshooting  trainer 

c.  AN/APQ-126  simulator 

d.  Omnidata  simulator 
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9.  Abstract:  This  report  reviewed  past  and  present 
applications  of  simulation  to  maintenance  training, 
concentrating  on  the  major  issues  and  problems  involved. 
Also,  the  report  identified  some  of  the  areas  where  further 
research  data  is  required.  The  report  is  organized  under 
six  major  headings: 

(1)  Simulation  Technology  as  Applied  to  Technical 
Training 

(2)  Application  of  Simulation  Technology  to  Technical 

Training:  Research  Findings 

(3)  Determination  of  Simulation  Requirements 

(4)  Design  and  Specification  of  Simulation  Requirements 

(5)  User  Acceptance  of  Maintenance  Simulators 

(6)  The  Cost-Effectiveness  of  Maintenance  Simulators 

A  brief  historical  overview  is  provided  under  the  first 
heading,  where  the  major  divisions  of  simulators  into 


General  Purpose  vs.  '  Systems-Specif ic,  2-dimensional  vs.  3- 
dimensional,  Prodecural  vs-.  Troubleshooting,  and  Electronic 
vs.  Non-Electronic,  are  discussed. 

Under  the  heading  of  "Application  of  Simulation  Technology 
to  Technical  Training:  Research  Findings,"  the  authors 
emphasize  the  demonstrations  of  effectiveness  of 
generalized,  interactive,  computer-driven  trainers  of 
moderate  to  low  physical  fidelity,  and  also  of  low-fidelity 
aids  and  mock-ups  which  are  not  considered  "simulators." 

The  section  headed _ "Determination  of  Simulation 
Requirements"  summarizes  methodology  and  problems  of 
incompleteness  of  information,  of  management,  and  of 
transf er-of-training  specifications.  Transf er-of-Training 
requirements  and  therefore  training  media  vary  with  stages 
of  learning,  with  low  fidelity  aids  or  devices  being  usually 
more  cost-effective  at  initial  stages  of  learning. 

In  the  section  headed  "Design  and  Specification  of 
Simulation  Requirements",  it  is  observed  that  a  body  of 
knowledge  still  does  not  exist  to  predict  the  effectiveness 
of  a  training  device.  Regarding  fidelity  requirements,  it 
is  agreed  that  functional  or  environmental  fidelity  is  of 
paramount  importance,  but  much  disagreement  exists  over 
physical  or  equipment  fidelity  requirements.  "There  now 
seems  to  be  no  question  but  that  training  devices  which 
possess  a  reasonable  level  of  functional  validity  but  which 
have  a  low  physical  fidelity  can  be  very  effective....  It 
appears  also  that  a  point  will  be  reached  beyond  which 
increases  in  either  functional  or  physical  fidelity  will  not 
increase  training  effectiveness." 

Intentional  deviation  from  realism  is  desirable  to  enhance 
the  teaching  capability  of  simulators  (in  such  features  as 
augmented  feedback,  etc.). 

The  role  of  various  groups  (e.g.,  instructors,  engineers, 
and  phsychologists)  in  decision-making  is  also  discussed  in 
this  section. 

In  the  section  headed  "User  Acceptance  of  Maintenance 
Simulators"  it  is  noted  that  instructors  underuse  and  misuse 
simulators  because  of  (1)  lack  of  knowledge  of  instructional 
techniques  and  their  proper  employment  through  features  of 
the  device;  (2)  concern  that  they  as  instructors  are  being 
replaced  rather  than  aided  by  the  device;  (3)  distrust  of 
devices  which  are  not  maximally  realistic,  i.e.,  lacking  in 
face  validity.  These  are  chiefly  management  problems. 

Under  "The  Cost  Effectiveness  of  Maintenance  Simulators, " 
the  authors  assert  that  "the  published  facts  indicate  that 


the  advantages  to  be  gained  through  the  use  of  maintenance 
simulator  remains  a  potential;  the  potential  gains  have  yet 
to  be  translated  into  practical  impacts."  This  results  from 
overtraining,  and  failures  to  develop  optimal  training 
programs  to  exploit  the  advantages  of  the  simulators. 

A  number  of  "Suggestions  for  Future  Research"  are  made  at 
the  end  of  the  report.  Among  other  suggestions,  a  new 
transfer-of- training  research  methodology  is  recommended,  to 
assess  the  benefits  of  new  simulators  more  accurately  and 
reliably. 
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5.  Short  Summary:  A  review  of  a  number  of  studies  of 
factors  affecting  transfer  of  training  from  flight 
simulators  to  aircraft  shows  that  transfer  is  generally 
proportional  to  the  degree  of  task  fidelity  and 
environmental  feedback  and  cuing  realism  in  the  simulator. 

A  general  scientific  theory  which  accurately  predicts 
optimal  fidelity  needed  for  maximal  transfer  of  flight 
training  has  not  yet  been  formulated. 

6.  Devices:  Aircraft  and  Spacecraft  Flight  Simulators 

7.  Institutions:  Defense  Technical  Information  Center 

8.  Type  of  Article:  Review 

a.  Number  of  Groups:  various  experiments  reviewed 

b.  Description  of  Groups: 

(1)  Subjects:  aviators  and  astronauts 

(2)  Controls:  not  applicable 

c.  Tests  or  Trials/Timing:  Varied 

d.  Number  of  Different  Types  of  Measures:  Varied 

e.  Description  of  Measurements  and  Ratings:  Varied 

f.  Experimental  Setting/Training  Context:  Varied 

g.  Statistical  Methods:  Varied 

h.  Variables  Being  Manipulated: 

(1)  Training  Devices:  Aircraft  and  Spacecraft 
Flight  simulators  of  varying  fidelity 

(2)  Fidelity  Levels:  High,  Medium,  Low  _ 
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(3)  Type  of  Task/Skill  Required:  Aircraft  Flight 

(4)  Task  Difficulty:  High 

i.  Stage  of  Training:  Varied 

j.  Trainee  Sophistication:  Varied 

k.  Incorporation  of  Device  into  P.O.I.: 

l.  User  Acceptance  or  Attitude:  Not  discussed 

m.  Use  of  Instructional  Features:  Varied 
Abstract : 

a.  Study  Synopsis:  A  review  of  simulation  training 
issues  and  then-available  studies  was  made  to  determine 
(A)  what  specific  elements  of  simulation  fidelity  (or 
"realism")  are  pertinent  to  the  design  and  function  of 
training  devices;  (B)  what  types  of  simulation 
training  devices  were  then  available;  (C)  in  what  ways 
various  available  devices  and  programs  answered  the 
need  for  specific  elements  of  fidelity. 

The  heart  of  the  review  discussed,  and  cited 
experiments  relevant  to,  the  following  questions: 

(1)  How  "true"  should  be  the  visual  input  which  is 
presented  to  the  pilot  in  simulated  situations? 

(2)  How  closely  should  the  feedback  from  the 
controls  resemble  that  of  the  aircraft? 

(3)  Are  motion  cues  necessary  for  operational 
simulation? 

(4)  What  environmental  factors  should  be  included? 

(5)  What  measures  are  available  to  determine 
quantitatively  the  effect  which  fidelity  of 
simulation  has  on  the  transfer  of  training? 

In  addressing  the  last  question,  the  author  noted  that 
quantitative  information  about  the  needed  degree(s)  of 
fidelity  was  scarce,  and  discussed  (a)  subjective  pilot 
ratings;  (b)  pilot  transfer  functions;  (c) 
physiologic  measurements;  and  (d)  proficiency 
measurements,  as  useful  but  largely  imperfect 
indicator s--the  last  simply  because  the  functions  which 
determine  pilot  proficiency  in  the  aircraft  had  never 
been  measured. 

b.  Results:  A  survey  of  a  number  of  studies  of 
factors  affecting  transfer  of  training  from  flight 
simulators  to  aircraft  showed  that  transfer  is 
generally  proportional  to  the  degree  of  task  fidelity 


and  environmental  and  cuing  realism  in  the  simulator. 

c.  Author's  Conclusions:  Although  part-task 
simulators  are  usually  less  expensive  and  of  lower 
fidelity  than  whole-  task  simulators,  they  can  be  very 
useful  for  the  learning  of  specific  tasks.  Their 
shortcomings  can  be  traced  back  to  the  lack  of 
fidelity,  particularly  in  simulating  motion.  The 
whole-task  flight  simulator  derives  its  advantages  as  a 
training  device  primarily  from  the  incorporation  of 
motion  cues,  if  the  addition  of  complex  motion  vectors 
increases  the  fidelity  of  the  situation  and  does  not 
result  in  spurious  stimuli.  It  is  the  psychologic, 
physiologic,  and  operational  realism  which  determine 
fidelity  in  simulation  and  not  face  validity  based  on 
physical  similarity  of  the  devices. 
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5.  Short  Summary:  This  is  both  a  theoretical  and  a  review 
paper  which,  with  reference  to  many  sources  in  the 
literature  of  simulation  training,  seeks  to  establish  a 
rigorous  and  useful  definition  of  simulator  fidelity.  It 
then  applies  the  definition  to  several  key  factors  in 
determining  fidelity,  and  finally  proposes  a  research 
strategy  to  investigate  empirically  the  effectiveness  of 
various  types  of  fidelity  configurations. 

6.  Devices:  No  specific  devices  discussed 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  (PERI-OU) ,  5001 
Eisenhower  Ave. ,  Alexandria,  VA  22333 

b.  Performing  Organization:  Same 

8.  Type  of  article:  Conceptual 

9.  Abstract:  This  paper  addresses  two  major  problems  in 
the  understanding  of  simulator  training  effectiveness: 
first,  given  that  the  fidelity  of  the  simulator — or  degree 
of  its  similarity  to  the  operational  environment--is  a  key 
ingredient  in  effective  training,  what  is  a  consistent, 
unambiguous  definition  of  fidelity  that  can  be  meaningfully 
applied  to  training  devices;  secondly,  given  such  a 
definition,  how  can  researchers  determine  the  kind  and 
degree  of  fidelity  necessary  to  produce  the  desired  training 
effect  in  a  cost-effective  way? 

Many  articles  are  reviewed  in  which  differing  concepts  of 
fidelity  have  been  proposed,  or  employed,  by  various 
researchers  and  contractors.  "Equipment  fidelity," 
"environmental  fidelity,"  "psychological  fidelity,"  "task 
fidelity,"  "behavioral  fidelity,"  "perceptual  fidelity," 
"functional  fidelity,"  "physical  fidelity"  are  all  labels 
that  have  been  used  with  varying  shades  of  meaning  that 
overlap  and  conflict  with  each  other  in  confusing  ways.  Out 
of  this  jumble,  the  author  selects  two  terms  which  are 
rigorous,  in  applying  exclusively  to  characteristics  of  the 
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equipment  rather  than  to  behaviors,  and  useful,  in  yielding 
logical  distinctions  between  two  separately  influential 
dimensions  of  fidelity:  these  are  "physical  fidelity"  and 
"funcional  fidelity":  "Fidelity  is  the  degree  of  similarity 
between  the  simulator  and  the  equipment  which  is  being 
simulated.  It  is  a  measurement  of  the  physical 
characteristics  of  the  simulator  (physical  fidelity)  ^nd  the 
informational  or  stimulus  and  reponse  options  of  the 
equipment  (functional  fidelity)." 

Reasons  for  departures  from  high  fidelity  in  a  simulator  are 
summarized  as  (1)  to  incorporate  "unrealistic"  training 
features  such  as  immediate  feedback  or  enhanced  stimulus 
cues,  which  aid  in  training  but  are  not  found  in  the  actual 
equipment;  (2)  to  lower  costs;  (3)  to  reduce  risk  of 
injury  or  equipment  damage;  (4)  technological  limitations 
prevent  achievement  of  perfect  imitation. 

The  central  question,  either  in  deliberately  departing  from 
high  fidelity  in  the  interest  of  training  efficiency  and 
cost-effectiveness,  or  in  accepting  an  unavoidable 
impairment  of  fidelity  due  to  technological  limitations,  is, 
how  do  variations  in  fidelity  influence  the  training 
effectiveness  of  the  device?  This  question  is  complicated 
by  modifications  due  to  many  elements  of  the  total  training 
context.  Among  the  more  influential  of  these  elements  are: 

(a)  Type  of  Task.  For  example,  fidelity  requirements 
in  teaching  the  cognitive  processes  in  troubleshooting 
decision-making  can  differ  from  fidelity  requirements 
in  psychomotor  operation  of  equipment. 

(b)  Stage  of  Learning.  Research  suggests  that  lower 
fidelity  is  relatively  more  effective  in  earlier  stages 
of  learning. 

(c)  Trainee  Abilities.  High  fidelity  is  wasted  on 
trainees  who  lack  aptitude  for  the  task. 

(d)  Simulator  Use  within  Training  Program,  and 
Attitudes  of  Instructors  and  Students. 

The  relevance  of  psychological  principles  of  learning  to 
fidelity  is  discussed,  in  terms  of  the  need  for  the 
trainee's  knowledge  of  results,  stimulus-response  linkage, 
motivation,  and  the  phenomenon  of  perceptual  constancy. 

To  establish  general  guidelines  for  determining  optimum, 
cost-effective  fidelity  levels,  empirical  investigation  of 
the  relationship  between  level  of  fidelity  and  training 
effectiveness  is  required.  The  best  avenue  of  investigation 
is  transfer  of  training  research,  but  this  work  is 
expensive.  To  maximize  the  efficiency  of  this  research,  the 
following  recommendations  are  made: 

(a)  Start  by  investigating  lower  levels  of  fidelity  and 
work  upward;  quoting  Eddowes  (1978)  "'as  soon  as  the 


capabilities  of  a  simulator  support  practice  of  a  set 
of  training  tasks,  stop  adding  to  it.  Everything  else 
the  simulator  does  will  cost  more,  and  while  it  may  not 
detract  from  training  effectiveness,  it  probably  won't 
add  to  it.  '  " 

(b)  Use  a  two-factor  research  model  which  isolates  the 
effects  of  physical  fidelity  and  functional  fidelity  to 
the  greatest  extent  possible.  A  series  of  studies 
which  used  this  approach  is  cited  as  an  example. 

(c)  Using  the  two-factor  approach,  develop  scales  which 
would  quantify  degrees  of  fidelity  and  would  be  applied 
objectively  to  various  types  of  equipment. 

In  summary,  the  author  states,  "the  proposed  definition  of 
fidelity  and  the  research  strategy  which  uses  this 
definition  should  enable  us  to  quantify  the  concept  of 
simulator  fidelity  for  any  given  task.  We  may  then  use  this 
data... to  prescribe  simulator  device  characteristics.... 

Our  efforts  will  pay  off  in  more  cost  effective  programs  of 
instruction. ..." 
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1.  Author:  Hays,  Robert  T. 

2.  Title:  Summary  of  Major  Ideas  from  Thorpe's 
Presentation,  "Constraints  on  Fidelity" 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  547, 
"Research  Issues  in  the  Determination  of  Simulator  Fidelity: 
Proceedings  of  the  ARI  Sponsored  Workshop  23-24  July  1981" 
November  1981;  Robert  T.  Hays,  editor,  pp.  57-64 

4.  Topic  Keywords:  Engineering  Simulation  ; 

Training  Simulation  ;  Fidelity  . 

5.  Short  Summary:  Several  points  are  summarized  which 
concern  the  definition  and  capabilities  of  simulation,  the 
context  of  fidelity  research,  and  the  constraints  placed  on 
the  training  community  by  a  single  "high  fidelity"  view  of 
fidelity. 

6.  Devices:  None  discussed 

7.  Institutions: 


a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
Alexandria,  VA  22333 

b.  Performing  Organization:  same 


8.  Type  of  Article:  Theoretical 


9.  Abstract:  This  paper  summarizes  points  made  in  a 
presentation  by  Major  Jack  Thorpe  of  the  Defense  Advanced 
Research  Projects  Agency  relating  to  fidelity  of  simulation 
in  training  devices.  The  most  prominent  points  are  as 
follows : 

(1)  It  is  important  for  researchers  in  simulation 
training  to  distinguish  clearly  between  engineering 
simulation  and  training  simulation.  Engineering 
simulation  requires  the  highest  fidelity  possible  to 
the  operational  situation;  cost-effective  training 
simulation  requires  only  the  degree  of  fidelity 
necessary  to  elicit  the  correct  learning  behavior  in 
the  student. 

(2)  Since  training  simulators  exist  in  a  total  training 
system,  interrelated  with  other  components  of  the 
system  (e.g.  instructor,  texts,  other  media),  a  change 
in  any  other  system  component  may  affect  the  simulator 
and  the  requirements  for  fidelity. 

(3)  A  mix  of  training  devices  may  provide  the  most 
effective  training.  (4)  Acceptance  of  simulation 
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training  may  require  a  public  relations  effort,  and  the 
effect  of  trainee  acceptance  of  simulators  on  transfer 
of  training  needs  to  be  researched. 

(5)  Highest  fidelity  does  not  always  produce  the  most 
effective  training;  "generalizations  based  on 
impressions  of  fidelity  or  on  single  cases  may  not 
apply  to  all  types  of  simulation  or  all  training 
contexts. " 

(6)  Skills  that  can  be  trained  on  simulators  fall  into 
three  main  types:  (a)  "substitution  skills"  which  can 
be  trained  on  the  actual  equipment;  (b)  emergency 
procedures  and  tasks -too  dangerous  to  be  practiced  on 
actual  equipment;  (c)  skills  that  are  only  practiced 
on  the  actual  equipment  during  war. 

(7)  "Fidelity"  is  not  synonymous  with  "high  fidelity". 
Device  fidelity  is  one  ingredient  of  a  total  training 
system,  and  creative  variations  in  instructional 
strategies  may  optimize  training  even  in  conditions  of 
low  fidelity. 
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1.  Author:  Hays,  Robert  T. 

2.  Title:  Training  Simulator  Fidelity  Guidance:  The 
Iterative  Data  Base  Approach 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  545, 
September  1981 

4.  Topic  Keywords:  Task  Analysis  ;  Simulator  Fidelity  ; 
Iterative  Data  Base  ;  Instructional  Systems  Development  . 

5.  Short  Summary:  Guidelines  are  proposed  for  creating  an 
interactive  data  base  for  determining  simulator  fidelity 
requirements  in  the  instructional  systems  development  (ISD) 
process.  The  paper  includes  an  explication  of  the  way  task 
analysis  information  is  to  be  translated  into  fidelity 
decision-  making,  and  emphasis  is  placed  on  the  need  for  an 
empirical  determination  of  the  effects  of  both  additive  and 
subtractive  fidelity  degradation. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Ave., 
Alexandria,  Va  22333 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  A  reliable,  unified  research  data  base  for 
determining  the  required  fidelity  of  simulation,  or  degree 
of  similarity  to  actual  equipment,  is  needed  to  help  design 
and  construct  cost-effective  training  simulators.  This 
paper  attempts  to  provide  a  preliminary  organizational 
framework  for  a  fidelity  data  base.  A  working  definition  of 
training  simulator  fidelity  is  presented  which  separates 
fidelity  into  two  dimensions  which  are  characteristics  of 
the  hardware  itself:  physical  and  functional. 

The  paper  discusses  three  stages  which  underlie  the 
organizational  framework.  First,  "the  issue  of  determining 
the  minimum  required  fidelity  for  a  training  simulator 
(is)...  located  in  its  proper  place  within  the  context  of 
Instr uct l onsl  Systems  Development....  Secondly,  the 
necessary  information  inputs  to  the  fidelity  decision 
process...  should  be  derived  from  a  task  analysis.... 
Finally,  a  proposea  structure  for  making  fidelity  decisions 
(is)...  presented." 
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1.  Authors:  Hritz,  Rohn  J.,  Harris,  Hobart  T  , 

Smith,  Jennifer  A.,  &  Purifov,  George  R. ,  Jr. 

2.  Title:  Maintenance  Training  Simulator  Design  and 
Acquisition--Handbook  of  ISD  Procedures  for  Design  and 
Documentation  -  Vol.  1 

3.  Source:  Draft  Tech.  Rep.  AFHRL  Contract  No. 
F33615-78-C-0019 

4.  Topic  Keywords:  Transfer  of  training  ;  Fidelity  ; 
Maintenance  Simulation  . 

5.  Short  Summary:  This  handbook,  using  flow  charts  and 
worksheets  as  aids,  comprehensively  guides  the  process  of 
making  decisions  on  maintenance  simulator  design  and 
acquisition.  A  method  is  presented  for  determining  training 
requirements,  deciding  how  to  meet  them,  and  selecting 
fidelity  levels  and  feature  options. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Air  Force  Human  Resources  Laboratory  , 
Technical  Training  Division,  Lowry  AFB ,  CO  80230 

b.  Performing  Organization: 

Applied  Science  Associates,  Inc.,  Valencia,  PA  16059 

8.  Type  of  Article:  Methodological 

9.  Abstract:  This  study,  one  of  series  of  related  studies 
in  instructional  systems  design,  provides  a  method  for 
deriving  maintenance  training  requirements  and  making 
training  equipment  fidelity  and  instructional  features 
design  decisions.  Three  major  decision  areas  are  covered: 

(1)  identifying  which  training  requirements  need 

har dwar e- ty pe  media;  (2)  the  degree  of  fidelity  of 
equipment  components;  (3)  what  instructional  features  are 
essential . 

All  procedures  in  this  report  use  decision  flow  charts  to 
increase  objectivity  and  to  maximize  the  consistency  and 
thoroughness  of  the  decision-making.  To  document  all 
decisions,  forms  and  worksheets  have  been  provided. 

The  roughly  chronological  order  of  the  steps  in  tne  decision 
and  validation  sequence  is  as  follows: 

a.  Identify  system  maintenance  requirements 

b.  Identify  characteristics  of  target  population 

c.  Determine  training  requirements 


1.  Author:  Hritz,  Rohn  J. 

2.  Title:  Effectiveness  Issues 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  547, 
"Research  Issues  in  the  Determination  of  Simulator  Fidelity: 
Proceedings  of  the  ARI  Sponsored  Workshop  23-24  July  1981," 
November  1981;  Robert  T.  Hays,  editor,  pp.  83-88 

4.  Topic  Keywords:  Training  Effectiveness  ; 

Simulation  Fidelity  . 

5.  Short  Summary:  Procedural  issues  in  the  development  of 
simulators  and  simulation  training  programs  are  discussed, 
with  an  emphasis  on  the  need  for  finding  new  methods  of 
evaluating  training  effectiveness. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
Alexandria,  VA  22333 

b.  Performing  Organization:  same 

8.  Type  of  article:  Theoretical 

9.  Abstract:  This  brief  report  discusses  procedural  issues 
in  the  development  of  simulators  and  simulation  training 
programs.  Design  procedures  begin  with  the  front-end 
analysis  usually  performed  at  the  Tech  Sergeant  level,  and 
comprises  the  synthesis  of  an  entire  training  system  as  well 
as  the  design  of  the  training  device(s),  which  comes  down  to 
making  separate  fidelity  decisions  on  each  component.  There 
are  losses  of  information  during  the  process,  especially 
where  the  procurement  specification  intervenes  between  the 
front-end  analysis  and  the  design  of  the  device.  Evaluation 
studies  have  failed  to  trace  results  back  to  the  decision 
models  that  were  used  in  determining  fidelity  and 
instructional  features. 

Another  shortcoming  of  device  effectiveness  evaluation  is 
the  reliance  of  evaluation  studies  on  comparisons  between 
alternative  training  systems.  Now  that  building  alternative 
training  systems  has  become  prohibitively  expensive,  new 
ways  of  evaluating  effectiveness  need  to  be  developed. 

An  existing  Instructional  Systems  Design  handbook  addressing 
these  issues  in  detail  is  referred  to. 
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The  author  points  out  that  "there  is  a  paucity  of  data  on 
the  relationships  between  simulator  fidelity  and  transfer 
effectiveness...  We  don't  know  very  much  about  how  some 
the  specific  features  and  characteristics  commonly  built 
into  simulators  contribute  co  or  detract  from  the  overall 
effectiveness...  If  simulators  are  to  be  widely  used  for 
training  and  profit  in  the  future  a  systematic  research 
program  must  be  conducted  to  establish  the  relationships 
between  transfer  of  training  and  physical  characteristics 
such  as  degree  and  fidelity  of  simulation." 
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1.  Author:  Hopkins,  Charles  0. 

2.  Title:  Simulators  for  Training  and  Profit 

3.  Source:  AFOSR-TR-77-0373 ;  ARL-76-10/AFGSR-76-5 ,  July 

1976 


4.  Topic  Keywords:  Transfer  of  Training  ; 
Incremental  Transfer  ;  Cost-Effectiveness  ; 
Fidelity  of  Simulation  ;  Platform  Motion  . 


1 

5.  Short  Summary:  A  discussion  of  the  cost-  effectiveness 
of  training  simulators  presents  a  transfer  vs.  cost  ratio 
model  for  rational  selection  and  use  of  simulators.  Cited 
data  show  a  high  negative  correlation  between  the  cost  and 
reliability  of  aircraft  simulator  equipment. 

, 

6  . 

Devices:  None  discussed 

7. 

Institutions : 

1  <*)• 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research 
(NL) ,  Bolling  AFB ,  DC  20332 

b.  Performing  Organization:  Aviation  Research 
Laboratory,  Institute  of  Aviation, 

University  of  Illinois,  Savoy,  IL  61874 

1 

8. 

Type  of  Article:  Theoretical 

1 

9. 

Abstract:  The  escalating  cost  of  simulators  without 

adequate  attention  to  cost-effectiveness  threatens  to 
produce  a  backlash  among  buyers.  One  particularly 
significant  cost  element--in  training  program  discontinuity 
as  well  as  in  maintenance  and  repair--is  the  lack  of 
reliability  in  costly,  high  fidelity  simulators.  This 
problem  is  often  overlooked.  Data  are  shown  which  indicate 
an  inverse  relationship  of  reliability  to  cost,  such  that  in 
terms  of  mean  flight  hours  between  failures,  for  example,  a 
$100,000  device  is  approximately  100  times  less  reliable 
than  a  $1,000  device. 

A  model  for  calculating  cost-effectiveness  developed  by 
Roscoe  (1971)  and  applied  to  an  experiment  reported  by 
Jacobs  and  Roscoe  (1975)  employs  an  Incremental  Transfer 
Effectiveness  Ratio  to  depict  the  relationship  between 
training  hours,  cost  of  simulator,  and  money  saved  or  lest 
by  using  the  simulator,  per  increment  of  ground  trair-  i 
time.  This  model  is  descriptive  rather  than  predictive: 
that  is,  the  ratios  used  are  derived  from 
transf er-of-training  experiments  on  the  equipment  under 
study . 


481 


The  advocacy  of  high  engineering  fidelity  in  simulators 
rests  on  three  arguments:  (1)  assumptions  that  the  greater 
the  likeness  between  a  simulator  and  the  transfer 
environment,  the  greater  the  transfer  of  training;  (2) 
trainees  are  more  highly  notivated  by  a  very  realistic 
simulator;  (3)  pilots  find  higher  fidelity  more  acceptable. 
The  first  argument  has  been  anaswered  by  research  which 
shows  that  not  all  degradations  in  fidelity  result  in 
degradation  of  transfer  performance  (the  reverse  is 
sometimes  true).  The  second  argument  is  beside  the  point; 
the  point  is  not  how  enthusiastically  the  trainee  flies  the 
simulator,  but  how  well  he/she  flies  the  real  aircraft  after 
simulator  training.  The  last  argument  may  be  dealt  with  in 
practice  by  "management  of  reinforcement  contingencies"; 
that  is,  by  finding  motivations  for  pilots  to  use  the 
simulator  apart  from  their  feelings  about  realism. 

If  the  training  community  fails  to  restrain  the  trend  toward 
unnecessary,  costly  increases  in  realism  in  simulators,  a 
backlash  against  all  simulators  may  result. 


1.  Author:  Hopkins,  Charles  0. 

2.  Title:  How  Much  Should  You  Pay  For  That  Box? 

3.  Source:  Human  Factors  ,  1975,  17(6),  533-541 

4.  Topic  Keywords:  Fidelity  of  Simulation  ; 
Cost-Effectiveness  ;  Flight  Simulation  . 

5.  Short  Summary:  The  high  cost  of  maximum  fidelity  in 
flight  simulators  has  not  been  experimentally  justified,  and 
personnel  in  the  simulator  training  field  must  anticipate 
eventual  backlash  from  the  trend  toward  costly  and  probably 
unnecessary  increments  in  realism. 

6.  Devices: 

a.  L-1011  (mentioned) 

b.  F-4J  (mentioned) 

7.  Institutions: 

a.  Sponsor:  Air  Force  Office  of  Scientific  Research  , 
Office  of  Naval  Research  , FAA,  and 

Air  Force  Avionics  Laboratory 

b.  Performing  Organization:  Aviation  Research 
Laboratory  (ARL) ,  Institute  of  Aviation, 

University  of  Illinois  at  Urbana-Champaign 

8.  Type  of  Article:  Review 

9.  Abstract:  "More"  is  not  necessarily  "better"  when  it 
comes  to  increasing  realism,  or  fidelity,  in  a  flight 
training  simulator,  if  transfer  of  training  to  the 
operational  aircraft  is  the  criterion.  First  of  all,  "the 
single  most  important  factor  that  influences  the 
effectivness  of  a  simulator  in  a  training  program  is  how  it 
is  used."  Secondly,  research  has  not  shown,  for  example, 
that  the  addition  of  such  "realistic"  features  as  platform 
motion  in  a  simulator,  once  generally  accepted,  enhances 
subsequent  trainee  performance  in  the  aircraft,  even  though 
it  may  enhance  performance  in  the  simulator.  Thirdly,  even 
if  the  addition  of  a  feature  enhances  transfer,  the  question 
of  its  cost-effectiveness  must  be  considered.  Studies  using 
a  comparative  cost  instrument  such  as  Roscoe's  (1972) 
Incremental  Transfer  Effectiveness  Ratio  are  needed  to 
vindicate  the  great  cost  increases  resulting  from  the 
attainment  of  extremely  high  fidelity.  "In  certain  cases 
there  may  even  be  a  negative  relationship  between 
effectiveness  and  the  inclusion  of  a  costly  feature." 


These  studies  focussing  primarily  on  verbal  rote 
memorization  yielded  little  evidence  of  general  ability 
enhancement. 

(2)  "Warm-Up",  in  which  a  subject  is  prepared  for 
performance  of  a  task,  such  as  verbal  recall,  by 
exercises  in  a  related  but  nonidentical  task,  has  seme 
implications  for  ability  training,  relating  to  the 
"set"  of  preparatory  postural  and  attentional 
adjustments  that  facilitate  performance. 

(3)  "Learning  to  Learn"  studies  have  given  some  support 
to  the  concept  of  nonspecific  transfer,  through  the 
inference  that  generalized  strategies,  or  "learning 
sets"  may  facilitate  performance  across  a  range  of 
tasks  within  a  certain  class.  At  least  one 
investigation  indicates  that  "learning  to  learn" 
phenomena  are  at  work  in  perceptual-motor  as  well  as 
verbal  learning. 

Studies  of  task  characteristics  and  conditions  of  practice 
and  the  use  of  cognitive  strategies  suggest  that,  depending 
on  the  kind  and  duration  of  practice,  and  the  amount  of 
abstraction  required,  nonspecific  transfer  has  taken  place 
between  different  tasks  in  verbal  and  pattern  recognition, 
when  the  performer  has  been  able  to  perceive  commonalities 
among  the  training  tasks. 

Applied  research  in  the  field  of  low  fidelity  simulation 
training  has  also  suggested  the  enhancement  of  nonspecific 
skills,  if  not  abilities.  Also  some  evidence  of  training 
originality  and  creativity  has  been  reported,  but  is  not 
conclusive;  research,  in  training  cognitive  abilities  has 
been  still  less  conclusive. 

This  review  suggests  that  there  is  some  indirect  evidence  to 
support  the  notion  that  abilities  can  be  trained.  Three 
important  considerations  from  research  support  this  idea: 

(a)  task  characteristics  tend  to  be  more  predictive  of 
transfer  than  just  the  training  materials  themselves; 

(b)  variability  of  training  or  practice  within  a  class 
of  response  types  facilitates  positive  transfer; 

(c)  transfer  may,  in  part,  be  mediated  by  a  strategy 
requiring  abstraction  of  important  features  in 
training. 

As  to  the  more  general  question  of  abilities  training,  a 
revision  may  be  in  order  for  the  traditional  conception  of 
"ability"  as  "a  fairly  enduring  trait,  which  in  the  adult  is 
relatively  difficult  to  change."  Conceivably,  this 
conception  of  ability  might  be  left  intact,  modified, 
however,  by  the  exploration  of  training  strategies  to  evoke 
the  potential  of  the  abilities  in  greater  degrees  of  power 
and  versatility. 
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1.  Author:  Hogan,  Joyce  C. 


2.  Title:  Trainability  of  Abilities:  A  Review  of 
Nonspecific  Transfer  Issues  Relevant  to  Ability  Training 

3.  Source:  Advanced  Research  Resources  Organization 
Technical  Report,  January  1978 

4.  Topic  Keywords:  Trainability  ;  Learning  to  Learn  . 

5.  Short  Summary:  A  review  of  literature  pertaining  to 
nonspecific  transfer  issues  finds  some  indirect  evidence 
that  abilities  can  be  trained.  It  concludes  that  transfer 
of  training  can  be  enhanced  by  greater  emphasis  on  task 
characteristics,  on  variability  of  training  within  a  class, 
and  on  a  mediation  strategy  that  requires  abstraction  of 
important  features  in  training. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Per sonnel  •  and  Training  Research  Programs, 
Office  of  Naval  Research  (Code  458) ,  Arlington,  VA 
22217 

b.  Performing  Organization:  Advanced  Research 
Resources  Organization,  4330  East-West  Highway, 
Washington,  DC  20014 

8.  Type  of  Article:  Theoretical/Review 

9.  Abstract:  "The  degree  to  which  basic  human  abilities 
are  trainable  is  a  question  posed  nearly  90  years  ago  that 
remains  relatively  unanswered."  This  is  one  of  the  author's 
conclusions  at  the  end  of  a  review  of  a  number  of  research 
efforts  and  hypotheses  that  have  touched  on  the  area  of 
nonspecific  training.  As  a  working  definition,  the  term 
"ability"  refers  in  this  paper  to  "empirically  determined 
relations  among  separate  observed  performances....  Skills 
are  more  specific,  and  define  levels  of  proficiency  in 
particular  tasks.  These  skills  can  be  described  in  terms  of 
component  abilities  required  to  perform  them,  with  certain 
abilities  shown  to  underly  performance  in  many  superficially 
different  human  tasks." 

With  this  in  mind,  the  author  surveys  studies  in  which 
nonspecific  transfer  of  learning  across  different  tasks  has 
been  investigated.  These  studies  fall  into  three  major 
categories : 

(1)  General  Memory  Ability  is  perhaps  the  earliest  line 
of  research,  beginning  with  William  James  in  1890. 
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equipment."  Simulator  training  is  seen  as  a  part  of  a 
program  which  also  includes:  classroom  lectures, 
books/slides/movies,  operational  demonstration,  operational 
equipment  practice,  and  on  the  job  training. 

While  simulator  training  problems  do  not  greatly  differ  from 
other  training  problems  in  a  functional  sense,  the 
uniqueness  of  simulator  problems  accrues  from  the 
complexity,  sophistication,  and  cost  of  simulation  equipment 
and  its  use.  In  the  interests  of  cost  effectiveness,  the 
design  of  equipment  must  emphasize  training  effectiveness 
rather  than  the  sheer  realism  of  the  device.  The  major 
question  to  answer  is,  "what  are  the  critical  skills  that 
the  training  device  is  supposed  to  facilitate?" 

The  following  topics  are  discussed  in  relation  to  training 
effectiveness:  (1)  establishment  of  training  goals;  (2) 

training  strategy  and  training  program  organization;  (3) 
instructional  methods  and  training  procedures;  (4)  learning 
of  perceptual  motor  skills;  (5)  performance  measurement. 

The  problem  of  performance  measurement  deserves  closer 
attention  than  is  generally  given.  Important  questions 
pertinent  to  performance  measurement  are:  what  to  measure, 
what  are  the  purposes  of  the  measurement,  reference  values 
for  error  measurement,  criteria  for  good  and  bad 
performance,  the  practical  importance  of  measurable 
differences,  and  the  question  of  equivalence  of  two  training 
methods  when  they  do  not  produce  reliable  differences  in 
performance. 

The  problem  of  measuring  transfer  of  training  reflects  the 
inadequate  understanding  of  the  relationship  between 
performance  in  training  and  performance  in  the  operational 
situation.  The  incremental  transfer  effectiveness  function 
is  recommended  over  the  Transfer  Effectiveness  Ratio  or  the 
Percent  of  Transfer  inasmuch  as  it  "reveals  the  training 
value  of  each  additional  hour  or  trial  of  training"  rather 
than  an  average  value  per  hour  or  trial. 

Research  and  development  of  a  theoretical  system  necessary 
to  define  and  measure  the  behavioral  processes  which  mediate 
between  task  and  device  variables  and  performance  have  not 
yet  been  done.  "If  the  necessary  fundamental  research  is 
performed  and  results  in  the  understanding  of  these 
intervening  processes,  then  I  believe  the  issues  of 
simulator  fidelity,  similarity  and  realism  will  disappear." 
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fidelity  issues  can  only  be  resolved  by  fundamental  research 
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system  to  relate  physical  task/device  variables  to 
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7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
Alexandria,  VA  22333 

b.  Performing  Organization:  Same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  In  this  paper  the  issue  of  fidelity  in 
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methodological  issues  posed  by  Gagne  twenty-seven  years  ago 
which  have  not  received  enough  attention  are:  the  structure 
of  skills  determinants  of  human  variability,  set, 
motivation,  and  transfer  of  learning;  and  task  behavior 
analyses  for  training,  proficiency  measurement,  and 
criterion  development. 

The  author  believes  that  simulation  research  "often  takes 
the  form  of  comparative  evaluation  of  different  levels  of 
simulator  technology."  Therefore  he  focusses  on  the  overall 
problems  of  training  within  which  the  design  of  simulation 
equipment  is  embedded.  "There  are  other  means  for  getting 
more  training  per  dollar  than  improving  simulation 
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ISD  process  produced  the  idea  of  employing  a  "training 
device  facilitator"  who  would  perform  a  role  analogous  to  a 
Department  of  Agriculture  extension  agent;  one  who  would 
communicate  with  device  users,  device  engineers,  researchers 
and  training  experts,  and  "provide  a  vital  communications 
link  between  individuals  who  may  only  be  concerned  with 
narrow  problems,  but  who  could  operate  more  effectively  with 
a  better  overview  of  the  whole  ISD  process. " 

Ideas  for  unifying  empirical  research  which  emerged  from  the 
workshop  were:  (1)  Taking  a  whole-system  approach;  (2) 
include  basic  research  on  the  mental  models  of  both  trainees 
and  instructors;  (3)  include  research  on  the  interaction  of 
levels  of  fidelity  and  device  configurations  with  a  large 
variety  of  variables  such  as  task  difficulty,  etc.;  (4) 
organize  research  to  produce  data  usable  by  diverse  groups; 
(5)  organize  so  that  later  efforts  can  build  on  previous 
research. 

The  relationship  between  fidelity  and  device  effectiveness 
remains  in  need  of  a  prescriptive,  predictive  model  which 
can  only  come  out  of  extensive  research  oriented  in  a 
unified  way  toward  that  goal.  Transfer  of  training  research 
was  agreed  upon  as  the  most  reliable  evaluative  tool,  but 
Transfer  of  Training  studies  so  far  have  not  yielded  either 
a  predictive  model  for  the  design  of  future  devices,  or  a 
method  to  evaluate  the  teaching  of  skills  on  a  simulator 
which  cannot  be  practiced  on  the  operational  equipment  and 
therefore  are  not  subject  to  transfer-of-  training  studies. 
Possibly  fruitful  areas  of  research  that  impinge  on  device 
effectiveness  were  discussed  under  the  headings  of  control 
strategies,  failures  in  simulator  training,  performance 
measurement,  and  simulator  instructors. 

Other  suggestions  coming  from  the  workshop  dealt  with  the 
use  of  new  instructional  technologies  by  instructors  and 
students,  ways  of  obtaining  funds  for  needed  research,  and 
criteria  for  prioritizing  among  research  issues. 


definition  of  fidelity,  represented  at  one  end  by  the  view 
that  only  the  physical  and  functional  dimensions  of 
f idelity--f idelity  as  hardware  specif ications--wer e  the 
essentially  meaningful  dimensions  to  evaluate,  represented 
at  the  other  end  by  the  view  that  multiple  types  of 
fidelity--  e.g.,  task,  perceptual,  etc. — needed  to  be 
discussed.  The  second  view  rejects  the  attempt  to  develop  a 
linear  metric  for  fidelity,  and  suggests  instead  using  a 
"branch-and-  flowchart  identifying  the  categories  and 
procedures  of  various  aspects  of  fidelity  which  apply  to  the 
situation."  Hays  recommends  that  "we  can  best  proceed  by 
dealing  with  fidelity  as  the  physical  and  functional  aspects 
of  the  training  device  and  treat  other  'types'  of  fidelity 
as  classes  of  variables  which  interact  with  these  aspects  of 
the  device  hardware." 

The  second  continuum  of  views  on  fidelity  was  of  the 
usefulness  of  the  fidelity  concept  itself;  at  one  end  was 
the  view  that  fidelity  was  a  non-issue,  at  the  other  the 
view  that  it  was  the  most  important  issue.  Three  objections 
to  placing  importance  on  the  question  of  fidelity  were 
brought  forward  in  the  working  group  sessions: 

(1)  Focussing  on  fidelity  takes  emphasis  away  from  the 
more  important  issues  of  "who  is  going  to  be  trained, 
for  what,  what  type  of  training  program,  what  type  of 
environment,  what  does  the  task  consist  of,  how  will  we 
determine  when  the  task  has  been  adequately  trained... 
if  you  deal  with  those  issues,  then  the  fidelity 
question  seems  to  be  taken  care  of." 

(2)  So  many  ways  of  measuring  fidelity  or  similarity 
exist  as  to  make  the  concept  meaningless  on  a 
theoretical  basis; 

(3)  Since  device  fidelity  requirements  change  as  the 
equipment  being  simulated  changes,  fidelity  is  too 
changeable,  over  time,  to  pin  down  specifically. 

Those  who  would  devalue  fidelity  recommended  the  following 
alternatives  for  increasing  training  device  effectiveness: 

(a)  historical  studies;  (b)  handbooks;  (c)  enhancing 
corporate  memory  or  experience;  (d)  altering  the 
reinforcements  for  designers  and  procurers;  (e)  emphasize 
back-end  analysis;  (f)  provide  examples  of  good  inexpensive 
training  devices;  (g)  anal^  al  techniques  to  define 
simulator  training  needs  for  engineers. 

Hays  maintained  that  the  pro  and  con  viewpoints  on  fidelity 
can  be  reconciled  by  treating  fidelity  as  a  summary  concept; 
"the  most  important  thing  to  remember  about  the  concept  of 
fidelity  is  to  use  it  as  a  tool,  but  not  to  let  it  dictate 
to  the  user  how  it  should  be  used.  " 

Discussion  of  communication  among  different  "players"  in  the 


472 


1.  Authors:  Hays,  Robert  T. ,  Holman,  Garvin  L. , 

Houston,  Thomas  J.,  Klein,  Gary  A.,  &  Swezey,  Robert  W. 

2.  Title:  Research  Issues  in  the  Determination  of 
Simulator  Fidelity:  Proceedings  of  the  ARI  Sponsored 
Workshop  23-24  July  1981:  Goals  and  Organization; 

Summaries  of  Topic  Areas  in  the  Working  Group  Sessions;  and 
Summary  Comments 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  547,  Robert 
T.  Hays,  editor;  pp.  1-8,  121-142,  159-166 

4.  Topic  Keywords:  Training  Effectiveness  ; 

Transfer  of  Training  ;  Simulation  Training  ; 

Simulator  Fidelity  . 

5.  Short  Summary:  In  the  ARI  sponsored  workshop,  a  range 
of  opinions  on  the  relationship  between  simulator  fidelity 
and  training  effectiveness  are  discussed,  and 
recommendations  are  made  as  to  how  to  perform  research  to 
clarify  these  issues. 

6.  Devices:  None  discussed 

7.  Institutions: 
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Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue, 
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9.  Abstract:  A  workshop  held  at  the  Army  Institute  in 
Alexandria,  VA  in  July  1981  was  attended  by  67  personnel 
including  Army,  Navy,  and  Air  Force  uniformed 
representatives,  and  civilian  representatives  from  industry, 
government,  and  academia.  "The  goal  of  the  workshop  was  to 
define  and  prioritize  research  issues  for  a  program  of 
empirical  research  on  the  relationship  between  simulator 
fidelity  and  training  effectiveness." 

This  synopsis  covers  the  portions  of  the  report  on  the 
workshop  comprising  the  summaries  of  topic  area  discussions 
in  working  group  sessions,  and  summary  comments  on  the 
entire  workshop  by  Robert  T.  Hays. 

In  his  summary  comments,  Hays  observes  that  workshop 
participants  viewed  the  fidelity  issue  along  two  different 
continua.  The  first  continuum  was  that  of  the  meaning  or 
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It  is  noted  that  specifying  training  fidelity  is  just  one 
portion  of  the  entire  Instructional  Systems  Development 
process,  taking  place  in  the  middle  development  phase,  and 
only  if  prior  design  decisions  have  selected  a  simulator  as 
the  training  medium. 

The  task  analysis  which  forms  a  first  step  in  the  process  is 
the  best  source  of  information  on  which  to  base  fidelity 
decisions.  Six  different  types  of  task  analysis  are 
identified,  of  which  any  one,  taken  alone,  is  ordinarily  not 
sufficient  for  making  all  necessary  decisions.  A  table 
listing  required  elements  of  task  analysis  output--which  may 
draw  on  several  types  of  task  analysis--is  given,  to  guide 
the  extraction  of  information;  "if  all  the  information... 
is  not  available  from  a  single  method  of  task  analysis, 
other  methods  should  be  employed  until  the  information  is 
obtained. " 

Other  tables  and  a  task  analysis  worksheet  are  shown,  to 
outline  and  illustrate  methods  for  translating  the  output  of 
the  task  analysis  into  specific  fidelity  recommendations. 
During  this  process  the  expert  must  make  decisions  based  on 
knowledge  of  the  effects  of  device  characteristics  on 
training  effectiveness.  "The  task  analysis  may  only  tell  us 
the  broad  outline  of  the  task  as  it  is  accomplished  in 
actual  field  conditions.  It  does  not  tell  us  how  to 
effectively  train  people  for  that  task."  This  is  where  the 
need  for  a  research  data  base  relating  fidelity  to  training 
effectiveness  is  essential.  There  are  no  simple 
relationships  between  fidelity  and  effectiveness;  variables 
such  as  stage  of  training,  task  difficulty,  and  others  all 
influence  the  relationship.  Thus  "the  proposed  simulator 
fidelity  data  base  should  contain  data  on  all  of  the 
variables  which  interact  with  simulator  fidelity  to  produce 
a  given  amount  of  training  effectiveness." 

Furthermore,  "reduced  simulator  fidelity  may  arise  from 
additions  to  as  well  as  subtractions  from  actual  equipment 
features.  Fidelity  research  must...  empirically  determine 
the  relative  effects  of  these  two  types  of  fidelity 
degradation...  additive  and  subtractive...." 

Finally,  the  author  recommends  that  "research  efforts  begin 
with  simple  questions....  Only  when  data  have  been  obtained 
from  the  simple  designs  should  more  complex  designs  be 
studied....  It  is  vital  that  the  empirical  data  be 
accumulated  in  a  systematic  manner." 
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d.  Determine  type  of  technical  training  materials 
required 

e.  Sequence  skills  and  knowledge 

f.  Identify  fidelity  and  simulated  features 

g.  Select  instructional  features 

h.  Prepare  ISD  specification 

i.  Identify  method 

j.  Prepare  course  control  documents 

k.  Prepare  instructional  materials  and  tests 

l.  Validate  instructions 

m.  Continue  training 

n.  Evaluate  training 

Under  each  step,  substeps  are  explained  in  concrete  detail, 
using  examples  where  helpful,  to  guide  the  designer  through 
all  tributaries  of  the  decision-making  system. 

This  study  emphasizes  the  importance  of  identifying  the 
skills  and  knowledges  required  as  parts  of  the  total  task 
which  is  being  trained.  An  explicit  taxonomy  of  skills  and 
knowledges  is  presented.  The  breakdown  of  tasks  into  skills 
and  knowledges  has  a  direct  bearing  on  fidelity 
requirements:  "Often  many  of  the  skills  and  knowledge  that 

make  up  a  task  do  not  need  to  be  taught  on  high-fidelity 
equipment  even  when  the  other  skills  and  knowledge  that  are 
part  of  the  f  >k  do. "  The  procedures  described  in  the 
handbook  assist  the  designer  to  match  fidelity  requirements 
with  specific  skills  and  knowledges,  and  thus  to  avoid  the 
costly  fidelity  "overkill"  that  results  from  analyzing 
fidelity  requirements  at  the  task  level  only. 

Further  steps  go  on  to  integrate  component  fidelity  needs 
found  at  the  skill  and  knowledge  level  into  the  task  level 
and,  further,  into  the  task-group  level. 

The  fidelity  question  is  subdivided  into  stimulus,  response, 
and  feedback  concerns;  a  flow  chart  is  provided  to 
illustrate  the  interrelationship  between  these  concerns. 

A  list  of  low-fidelity  alternatives,  e.g.,  mock-ups,  tapes, 
films,  is  also  provided. 
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5.  Short  Summary:  This  report  summarizes  activities  and 
results  of  a  project  which  produced  models  and  guides  for 
meeting  maintenance  training  requirements  with  appropriate 
training  devices.  The  procedures  for  determining  when  to 
use  a  simulator,  and  what  degree  of  simulation  it  should 
possess,  and  what  features  it  should  incorporate,  are 
discussed,  along  with  nine  problem  areas  and  recommended 
solutions  to  each. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization: 

Applied  Science  Associates,  Inc. ,  Box  158,  Valencia,  PA 
16059 

8.  Type  of  Article:  Review 

9.  Abstract:  This  report  summarizes  activities  and  results 
of  a  project  which  produced  models  and  guides  for  answering 
maintenance  training  requirements  with  appropriate  training 
devices.  The  report  (a)  describes  the  research  activities 
that  were  performed;  (b)  describes  the  products  and  reports 
produced  and  the  reactions  of  the  intended  audiences  to 
these  products;  and  (c)  presents  a  list  of  problem  areas, 
recommendations,  and  areas  for  future  research. 

The  basic  problem  being  addressed  was  the  inadequacy  of 
information  being  supplied  to  fabricators  of  training 
requirements.  It  was  felt  that  better  guidance  for 
Instructional  Systems  Development  (ISD)  analysts  and  device 
procurement  personnel  would  help  correct  these  inadequacies. 
The  research  activities  that  were  performed  focussed  on 
procedures  for  "designing  and  documenting  maintenance 
trainers:  i.e.,  the  procedures  for  determining  when  to  use 

a  simulator,  the  procedures  for  determining  the  degree  of 
fidelity  of  the  trainer  components,  and  the  procedures  for 
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selecting  and  defining  the  instructional  features  of  the 
maintenance  trainer." 

The  analysis,  clarification,  and  expansion  of  procedures 
resulted  in  four  documents:  (1)  "Maintenance  Training 
Simulator  Design  and  Acquisition:  Summary  of  Current 
Procedures"  AFHRL-TR-7 9-23 ,  November  1979;  (2)  "Maintenance 

Training  Simulator  Design  and  Acquisition:  Handbook  of  ISD 
Procedures  for  Design  and  Documentation";  (3)  "Maintenance 
Training  Simulator  Design  and  Acquisition:  ISD-Derived 
Training  Equipment  Design";  (4)  "Maintenance  Training 
Simulator  Design  and  Acquisition:  Prime  Development 
Specification  for  Maintenance  Training  Simulatore."  The 
model  or  generic  specifications  presented  in  the  last  three 
documents,  adaptable  to  all  kinds  of  maintenance  training 
devices,  provide  systematic  and  detailed  "fill  in  the 
blanks"  type  guidance  for  identifying  training  objectives, 
target  populations,  device  characteristics  and  fidelity 
requirements,  and  other  items  pertinent  to  device  design. 

Reactions  to  the  documents  cited  above  were  obtained, 
problem  areas  were  identified,  and  recommendations  were  made 
for  improvement.  The  nine  problem  areas  are  summarized  as 
follows:  (1)  Pressure  on  the  ISD  analyst  to  accelerate  the 

acquisition  cycle;  (2)  Lack  of  continual  communication 
between  ISD  and  device  procurement  personnel;  (3)  transfers 
of  ISD  personnel  before  their  expertise  has  been  fully 
realized;  (4)  need  for  better  exposure  to 

"state-of-the-art"  devices;  (5)  translation  of  engineering 
changes  in  operational  equipment  to  the  trainer;  (6)  need 
to  integrate  different  procedural  handbooks  into  one 
document;  (7)  need  to  design  contractor-provided  data  base; 
(8)  need  for  Instructional  Features  scenarios ;  (9) 

specialists/experts  should  closely  review  certain  parts  of 
the  device  specifications  before  they  are  submitted  to  the 
contractor . 
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documenting  training  equipment  designs  derived  from  an 
Instructional  Systems  Development  analysis.  The  model 
standardizes  the  communication  between  ISD  and  Simulator 
System  Program  Office  Personnel.  The  authors  outline  a 
method  for  communicating  a  training  device  design  to  the 
simulator  system  program  office  once  the  need  for  such  a 
device  has  been  established  by  ISD  analysis. 
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9.  Abstract:  This  technical  paper  presents  a  model  for 
standardizing  communications  between  Air  Force  Instructional 
Systems  Development  (ISD)  personnel  and  Simulator  System 
Program  Office  (SIMSPO)  personnel,  in  the  design  of 
maintenance  training  devices.  The  body  of  the  paper 
outlines  training  device  design  data  requirements  in  a 
systematic  way  so  that  all  the  ISD  analyst  need  do  is  fill 
in  the  blanks  provided.  An  appendix  supplies  instructions 
for  applying  the  model.  Using  the  model,  ISD  analysts  will 
specify  such  design  information  as:  (1)  characteristics  of 
the  target  population  and  who  will  use  the  trainer;  (2)  a 
list  of  the  training  objectives  to  be  achieved  using  the 
trainer;  (3)  a  list  of  the  tasks  to  be  practiced  and/or 
acquired  on  the  trainer  and  a  list  of  malfunctions  to  be 
presented;  (4)  a  scenario  discussing  how  the  trainer  will 
be  used  to  achieve  the  objectives;  (5)  a  list  of  the 
physical  and  functional  characteristics  of  the  components  to 
be  represented  on  the  trainer  (with  illustrations  where 
possible);  (6)  a  description  of  the  instructional  features 
required  on  the  trainer.  Some  of  this  information,  such  as 
specifications  of  the  characteristics  of  the  target 
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population,  are  optional. 

It  is  recognized  that  completion  of  the  design  is  an 
iterative  process  and  that,  in  view  of  real-world  time 
constraints,  the  model  may  be  completed  in  stages. 

The  decision-making  procedures  which  l^ad  to  the  design 
specifications  in  the  model  are  not  contained  in  this  paper, 
but  rather  in  other  ISD  procedural  handbooks,  in  particular 
"Maintenance  Training  Simulator  Design  and  Acquisition-- 
Handbook  of  ISD  Procedures  for  Design  and  Documentation"  by 
Hritz,  Harris,  Smith,  and  Purifoy,  AFHRL-TR-81-51 ,  1980. 
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1.  Authors:  Hritz,  Rohn  J.,  Purifoy,  George  R.  Jr.,  & 
Fitzpatrick,  Jean 

2.  Title:  Prime  (Item)  Development  Specifications  for 
Maintenance  Training  Simulator:  Model  Specifications  Volume 
I  &  II,  final  draft 

3.  Source:  Applied  Science  Associates,  Inc.,  Valencia,  PA 
16059,  April  1981 

4.  Topic  Keywords:  Maintenance  Training  ; 

Maintenance  Simulation  ;  Training  Devices  . 

5.  Short  Summary:  This  procedural  guide  provides  a  generic 
specification  format  that  can  be  adapted  to  specification  of 
requirements  for  any  type  of  maintenance  trainer  and  is 
tailored  for  use  by  the  Air  Force  System  Program  Office. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization: 

Applied  Science  Associates,  Valencia,  PA  16059 

8.  Type  of  Article:  Methodological  (a  procedural  guide) 

9.  Abstract:  This  report  contains  a  model  or  generic 
specification  for  maintenance  training  equipment.  It  is 
divided  into  two  volumes:  Volume  I  contains  the  model; 
Volume  II,  the  Appendix/Handbook,  contains  instructions  for 
tailoring  the  paragraphs  and  subparagraphs  in  the  model 
specification  for  a  specific  application.  Volume  I  has 
blanks  to  be  completed  by  personnel  at  the  Air  Force 
Simulator  System  Program  Office;  Volume  II  provides 
instructions  for  completing  the  blanks.  The  model 
specification  contains  paragraphs  and  subparagraphs  which 
specify  both  training  requirements  and  engineering 
requirements . 

Contents  of  Volume  I  include  a  total  of  approximately  200 
line  items  of  specifications  under  the  following  headings: 
(General)  Description,  Characteristics,  Design  and 
Construction,  Logistics,  Personnel  and  Training,  Quality 
Assurance  Program,  and  Preparation  for  Delivery. 

Contents  of  Volume  II  include,  for  each  of  the  approximately 
200  line  items  of  specifications,  Rationale  and  Guidance, 
Performance  Parameters,  Background  and  Sources,  and  Lessons 


Learned. 


These  specifications  are  best  used  in  combination  with 
documentation  prepared  by  an  Instructional  Systems 
Development  analyst  as  described  in  "Maintenance  Training 
Simulator  Design  and  Acquisition:  ISD-Derived  Training 
Equipment  Design,"  prepared  by  Hritz  and  Purifoy  (AFHRL-TR- 
81-52,  February  1982). 
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1.  Authors:  Huddleston,  H.  F.,  &  Rolfe,  J.  M. 

2.  Title:  Behavioral  Factors  influencing  the  use  of  flight 
simulators  for  training 

3.  Source:  Applied  Ergonomics,  September  1971,  pp. 

141-148 

4.  Topic  Ke^  ^ords:  Flight  simulation  ; 

Transfer  of  learning  . 

5.  Short  Summary:  A  review  article,  briefly  summarizing 
several  issues  in  the  use  of  flight  simulators  emphasizes 
the  role  of  a  simulator  as  a  training  aid  rather  than  total 
training  device.  At  the  same  time,  it  enumerates  the 
drawbacks  of  training  solely  in  flight. 

6.  Device:  None  discussed 
7  .  Insti tutions : 

a.  Sponsor:  Royal  Air  Institute  of  Aviation 
Medicine,  Farnborough,  Hampshire,  England 

b.  Performing  organization:  same 

8.  Type  of  article:  Review 

9.  Abstract:  This  paper  reviews  literature  on  the  use  of 
flight  simulators,  and  propounds  the  view  of  a  simulator  as 
a  training  aid  rather  than  a  total  training  environment. 

A  transfer  effectiveness  model,  and  the  influence  of  the 
training  organization  as  a  whole  on  the  simulator  training 
program  are  discussed.  Maximal  use  of  the  simulator  can 
conflict  with  the  need  to  keep  motivation  high  in  training; 
"the  problem  is. . .  to  determine  the  correct  ratio  of  flying 
hours  to  simulator  hours  in  order  that  training 
effectiveness  and  aircrew  motivation  (both)  be  preserved 
in...  training  environments." 

Demands  on  instructors  are  also  considered.  Device  training 
may  require  so  much  work  of  an  instructor  in  running  the 
simulation,  that  he/she  has  little  time  to  "instruct". 

Several  issues  that  bear  on  fidelity  requirements  are 
discussed.  There  is  a  need  for  research  to  more  fully 
determine  what  are  the  behavioral  strategies  used  in  real 
flight  which  need  to  be  elicited  by  the  simulator,  and  what 
are  the  sensory  threshholds  of  various  stimuli  that  cue  the 
behaviors.  Furtherfore,  research  is  needed  to  determine  the 
variations  in  proportions  of  sub-task  constituting  the 
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entire  workload,  as  function  of  flight  phase  and 
environmental  conditions. 

Information  flow  models  are  described  which  depict  the 
relationship  between  subject,  controller,  and  machine  via 
feedback  loops  in  the  simulator. 

The  requirement  for  perceptual  fidelity  in  the  device  is 
conceived  as  a  question  of  determining  the  point  of 
diminishing  returns:  "the  key  question  concerns  the  point 
at  which  we  stop  adding  items  of  simulator  hardware  which, 
taken  singly,  demonstrably  add  to  the  catalogue  of 
sensations  simulated." 
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1.  Authors:  Huff,  Edward  M. ,  &  Nagel,  David  C. 

2.  Title:  Psychological  Aspects  of  Aeronautical  Flight 
Simulation 

3.  Source:  American  Psychologist,  1975,  March,  30(3), 
426-439 

4.  Topic  Keywords:  Flight  Simulation  ; 

Research  Simulators  . 

5.  Short  Summary:  This  review  paper  discusses  theoretical 
design  issues  in  the  current  evolution  of  flight  simulators. 
It  focuses  on  ways  of  conceptualizing  and  measuring  pilot 
experiences  during  the  implementation  of  research  programs. 
It  is  suggested  that  the  theory  of  signal  detectability  and 
nonmetric  multidimensional  scaling  techniques  be  applied  to 
help  build  a  foundation  for  a  science  of  simulation. 

6.  Devices:  Several  devices  are  mentioned  with  respect  to 
design  features  but  not  with  respect  to  effectiveness. 

7.  Institutions: 

a.  Sponsor:  NASA 

b.  Performing  organization:  Man-Machine  Integration 
Branch,  NASA,  Ames  Research  Center,  Moffett  Field,  CA 
94035 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  paper  discusses  issues  in  the 
development  of  ground-based  flight  simulators  for  research 
purposes,  without  regard  to  training  effectiveness.  A 
difference  is  noted  in  technical  requirements  between 
research  simulators  and  training  simulators,  of  which  one 
aspect,  for  example,  is  the  generally  lower  degree  of 
cockpit  fidelity  in  research  devices,  due  to  the  need  for 
flexibility  in  simulating  a  variety  of  aircraft. 

The  principal  components  of  a  flight  simulator  are  discussed 
in  turn:  the  cockpit,  the  visual  subsystem,  motion 
subsystem,  and  computer  control  device.  In  the  discussion 
of  these  components,  the  following  issues  relevant  to 
unresolved  questions  in  both  research  and  training 
simulation  are  raised: 

(a)  Contradictory  data  have  been  found  regarding  the 
relative  contribution  of  visual  and  motion  cues  to 
pilot  performance  in  simulators.  It  is  unknown,  for 
example,  why  a  pilot  in  simulation  tends  to  land 
shorter  on  a  runway  than  in  real  flight;  one 
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experiment  suggests  that  insufficient  motion  cues  are 
more  to  blame  than  inadequate  visual  systems.  Two 
somewhat  differing  views  of  spatial  perception--  one 
emphasizing  the  interaction  between  optic  arrays,  the 
other  emphasizing  the  internal  interpretive  models  of 
the  observer--are  adduced  by  way  of  illumination,  but 
the  authors  note  that  no  systematic  tests  of  these 
notions  have  been  conducted. 

Empirical  studies  of  simulated  motion  effects  on  pilot 
performance  have  produced  ambiguous  findings.  We  do 
not  know  what  subtle  interactive  effects  motion  and 
vision  cues  may  have  on  the  complex  imformation 
processing  performed  by  the  pilots  of  simulated 
aircraf  t. 

(b)  The  impact  of  simulation  fidelity  (that  is,  the 
adequacy  of  perceptual  effects  induced  by  the  simulator 
as  a  result  of  engineering  and  construction)  on 
simulator  validity  (that  is,  the  general izability  of 
simulator  effects  on  the  pilot  to  the  real  flight 
situation)  is  based  on  a  prior  assumptions  which  place 
a  high  value  on  realism.  Many  researchers  feel  that 
simulation  validity  may  be  operationally  defined  by  the 
extent  of  which  simulated  system  performance 
corresponds  with  that  obtained  from  a  real  aircraft. 
Informed  pilot  opinion  is  also  relied  upon  as  a  measure 
of  validity.  However,  the  individual  pilot  has 
personal  biases  and  his  view  is  restricted  to  aircraft 
for  which  he  has  sufficient  experience  to  make  a 
comparison. 

Thus  it  is  a  fundamental  problem  to  assess  the  extent 
to  which  engineering  "improvements"  are  relected  in 
increased  psychological  realism.  Research  to  date  has 
been  primarily  concerned  with  the  effects  of  improved 
engineering  on  overall  simulator  validity,  rather  than 
witn  measurements  of  psycho-physical  relationships  and 
perceptual  processes  influenced  by  various  subsystem 
designs. 

(c)  Two  techniques  are  suggested  to  help  build  a 
foundation  for  a  science  of  simulation: 

(i)  The  theory  of  signal  detectability,  which  can  be 
applied  wherever  binary  decisions  are  made  in  the  face 
of  subjective  uncertainty; 

(ii)  Nometric  multidimensional  scaling  techniques, 
particularly  those  used  to  characterize  individual' 
differences  in  the  response  to  complex  perceptual 
stimuli. 


A  plea  is  made  to  develop  systematic  research  programs  to 
produce  principles  of  simulation  to  deal  with  these  and 
related  problems. 
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1.  Authors: 


Iffland,  Harold  L. ,  &  Whiteside,  George  A. 


2.  Title:  Aircraft  Simulator  Data  Requirements  Studv,  Vol. 
II 

3.  Source:  ASD-TR-77-25  Volume  II,  January  1977 

4.  Topic  Keywords:  Simulator  Data  Requirments  ; 

Flight  Simulators  ;  Air  Weapon  System  Design  Data  ; 
Computer-Aided  Design  . 

5.  Short  Summary:  A  survey  of  flight  simulator 
manufacturers,  simulator  acquisition  activities,  and 
aircraft  manufacturers,  yields  criteria  of  a  General 
Requirement  for  supplying  data  for  simulator  development. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Aeronautical  Systems  Division, 

Wr ight-Patter son  Air  Force  Base,  OH  45433 

b.  Performing  Organization: 

Systems  Research  Laboratories,  Inc. ,  2800  Indian 
Ripple  Road,  Dayton,  OH  45440 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  Impediments  to  the  flow  of  data  on  new 
aircraft  to  simulator  manufacturers,  and  suggestions  for 
overcoming  them,  constitute  the  subject  matter  of  this 
report.  The  contents  of  this  volume  are  summarized  in  the 
paper  by  Whiteside,  George  A.  entitled  "Aircraft  Simulator 
Data  Requirements  Study:  Executive  Summary"  ASD-TR-77-25 
Vol.  I. 

A  survey  of  flight  simulator  manufacturers,  simulator 
acquisition  activities,  and  aircraft  manufacturers  was 
performed  to  determine  a  General  Requirement  for  supplying 
the  necessary  data  for  simulator  development  on  a  schedule 
such  that  "the  simulator  can  be  on-site,  ready  for  training 
before  the  first  production  aircraft  is  delivered  to  the 
operating  command." 

Discussion  of  the  survey  touches  on  the  following  topics  and 
problems  in  the  acquisition  of  data  for  simulation: 
acceptance  test  procedures;  malfunction  and  abnormal 
operations;  barriers  to  communication  between  simulator 
manufacturer  and  aircraft  design  engineers;  timing  of 
physical  data,  aerodynamic  data  estimate,  system  data 
estimates,  "hot  bench"  tests,  and  dynamic  tests; 
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intelligence  data;  tactical  program  tapes  and  duplication 
of  aircraft  central  computers;  enforcement  of  the 
data-acquisition  schedule;  procurement  of  crew  station 
parts  from  the  aircraft  manufacturer;  project  pilots  and 
project  crew  members;  ana  role  of  instructor  pilots. 

A  number  of  specific  recommenations  are  made  to  facilitate 
the  supplying  of  useable  data  to  the  simulator  manufacturer. 
The  primary  recommendtions  are,  that  the  initial  contract 
for  aircraft  development  should  require  delivery  of  the  data 
needed  to  develop  a  full  mission  simulator,  and  that  the 
activity  should  follow  up  on  these  requirements  to  ensure  a 
timely  delivery  of  accurate  information.  The  Air  Force 
Flight  Test  Center  must  also  be  tasked  to  provide  handling 
qualities  and  performance  parameters  to  the  simulator 
manufacturer  as  early  as  possible  in  the  test  program;  the 
AFTC  should  also  assign  a  qualified  test  pilot  curent  in  the 
aircraft,  and  a  flight  test  engineer,  to  assist  in  the 
development  of  the  simulator. 


» 
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1.  Authors:  Isley,  Robert  N.  ,  Corley,  Winon  E. ,  & 

Caro,  Paul  W. 

2.  Title:  The  Development  of  U.S.  Coast  Guard  Aviation 
Synthetic  Training  Equipment  and  Training  Programs 

3.  Source:  HumRRO-FR-D6-7 4-4 ,  October  1974 

4.  Topic  Keywords:  Flight  Simulation  ; 

Rotary  Wing  Training  Requirements  ; 

Variable  Cockpit  Training  System  ;  Cost-Effectiveness  . 

5.  Short  Summary:  This  paper  describes  a  three-phase  study 
in  the  development  of  a  synthetic  training  and  academic 
training  program  for  U.S.  Coast  Guard  helicopter  pilots.  A 
bef ore-and-af ter  Cost  Benefit  comparison  shows  savings 
through  use  of  the  new  programs. 

6.  Devices:  Variable  Cockpit  Training  Devices  for  HH~2A 
and  HH3F  helicopters 

7.  Institutions: 

a.  Sponsor:  Department  of  Transportation, 

U.S.  Coast  Guard,  400  Seventh  Street,  S.W. ,  Washington, 
DC  20590 

b.  Performing  Organization: 

Human  Resources  Research  Organization  ,  300  North 
Washington  Street,  Alexandria,  VA  22314 

8.  Type  of  Article:  Review 

9.  Abstract:  This  paper  describes  the  latter  two  of  three 
phases  in  the  development  and  implementation  of  synthetic 
and  academic  flight  training  programs  for  the  U.S.  Coast 
Guard.  Programs  were  developed  using  simulators  for  two 
helicopters,  the  HH52A  and  HH3F,  with  a  focus  on  training 
qualified  pilots  to  perform  Sear ch-and-Rescue  Operations.  A 
qualification  program  for  fixed-wing  pilots  transitioning  to 
the  HH52A  was  also  developed,  along  with  an  Instrument 
Proficiency  program  for  operational  helicopter  pilots  in 
both  helicopters,  and  a  Transition  program  for  helicopter 
pilots,  principally  those  just  out  of  Undergraduate  Pilot 
Training,  to  learn  to  fly  the  two  helicopters. 

A  bef ore-and-af ter  Cost  Benefit  Comparison  is  given  which 
indicates  substantial  savings  through  use  of  the  new 
programs,  using  primarily  simulators,  in  contrast  with  the 
former  program  which  used  operational  equipment  in  training. 
It  was  calculated  that  the  cost  of  the  simulators  plus  the 
associated  training  programs  would  be  paid  back  within  14  to 
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1.  Author:  Miller,  Robert  B. 

2.  Title:  Handbook  on  Training  and  Training  Equipment 
Design 

3.  Source:  W ADC  Technical  Report  53-136,  June  1953 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Training  Effectiveness  ;  Simulation  ;  Knowledge  of  Results 
;  Learning  Feedback  ;  Action  Feedback  ; 

Motivation  Feedback  ;  Psychological  Set  ;  Overlearning  ; 
Stimulus-Response  . 

5.  Short  Summary:  The  handbook  provides  a  detailed  and 
comprehensive  discussion  of  issues  relevant  to  the  design 
and  use  of  training  equipment,  with  an  emphasis  on  applying 
the  psychological  principles  of  human  learning. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  United  States  Air  Force 

Wright  Air  Development  Center,  Air  Research  and 
Development  Command,  Wright-  Patterson  Air  Force  Base, 
OH  45433 

b.  Performing  Organization: 

American  Institutes  for  Research  Washington,  DC 

8.  Type  of  Article:  Theoretical/Methodological 

9.  Abstract:  This  339-page  book  (of  which  the  last  64 
pages  are  bibliography  and  index)  published  in  1953, 
provides  a  detailed,  comprehensive  discussion  of  issues 
relevant  to  the  design  and  use  of  training  equipment.  It  is 
addressed  principally  to  those  who  have  already  had  some 
training  in  the  psychology  of  human  learning.  An  outline  of 
the  content,  following  the  chapter  headings  listed  in  the 
book's  Table  of  Contents,  is  as  follows: 

Section  I.  Human  Learning  -  An  Overview 

a.  General  Adjustment  to  the  Situation 

b.  Responses  get  Learned  to  Generalized  Situation  Cues 

c.  Localization  of  Responses 

d.  Verbal  Mediation  between  Cue  and  Action 

e.  Context  Learning:  Response  In te r- Assoc  1  a t i on 

f.  The  Importance  of  "Overlearning" 

Section  II.  The  Role  of  the  Instructor  in  Training 

a.  Motivating  Students 

b.  Explaining  Principles,  Relationships 


The  second  program  was  the  development  of  a  Generalized 
Maintenance  Trainging  Simulator  (GMTS)  which  was  to  be  aimed 
at  equipment  familiarization,  simulated  practice,  and 
testing  on  specific  equipment  associated  with  a  particular 
class  C  school.  The  GMTS  is  a  2-dimensional  CRT  simulation 
with  which  the  student  interacts  by  means  of  a  touch  stylus. 
The  GMTS  was  being  programmed  for  use  as  a  trainer-simulator 
for  the  WSC-3  Satellite  Communications  System,  and  four 
prototypes  were  about  to  be  placed  in  the  school  for  testing 
and  evaluation. 

The  GMTS  system  embodies  a  single  universal  execution 
routine  for  simulating  equipments,  such  that,  when  moving 
from  one  system  simulation  to  another,  only  the  data  base, 
not  the  program,  need  be  replaced.  Subject-matter  experts 
at  schools  are  responsible  for  preparing  the  data  base. 

A  typical  troubleshooting  problem  is  talked  through  to 
illustrate  the  use  of  the  system. 

Among  the  advantages  cited  for  the  GMTS  is  the  ability  to 
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1.  Miller,  Knox  E. ;  Malec,  Vernon  M. 


2.  Title:  Simulation  in  Maintenance  Training 

3.  Source:  unpublished  Navy  Personnel  Research  and 
Development  Center  Report,  1980 

4.  Topic  Keywords:  Maintenance  Simulation  ; 

Generalized  Maintenance  Trainer  ; 

Electro-mechanical  Maintenance  Training  ; 

Electronics  Maintenance  Training  . 

5.  Short  Summary:  This  report  describes  three  maintenance 
training  simulation  research  programs  being  conducted  by  the 
Naval  Personnel  Research  and  Development  Center.  The 
programs  are  discussed  in  the  context  of  overall  naval 
maintenance  training  objectives. 

6.  Devices:  Generalized  Maintenance  Training  Simulator 

7.  Institutions: 

a.  Sponsor: 

()  •  Navy  Personnel  Research  and  Development  Center,  San 

Diego,  CA  92152 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 

9.  Abstract:  This  report  describes  three  maintenance 
training  simulation  research  programs  being  conducted  by  the 
Navy  Personnel  Research  and  Development  Center.  Two  of 
these  programs,  aimed  at  distinctly  different  levels  of 
electronics  maintenance  training,  were  well  under  way  at  the 
time  of  the  report,  with  a  device  prototype  approaching  the 
testing  stage  in  one  of  them.  The  third  program,  aimed  at 
creating  a  common  maintenance  training  data  base,  generic 
specifications,  and  an  applications  handbook,  was  in  the 
intial  planning  stage. 

The  first  program  was  the  Electronics  Equipment  Maintenance 
Training  (EEMT)  System,  which  was  "aimed  at  improving 
initial  skills  training  at  the  Navy  A  school  level."  The 
oesign  comprised  one  unit  intended  for  use  in  basic  skills 
training,  and  a  second  unit  intended  to  provide  hands-on 
experience.  The  first  unit  consisted  of  a  CRT  interactive 
display,  an  image  pro3ector,  and  a  touch  pen  for  student 
interactions,  with  a  minicomputer  at  the  heart  of  the 
system.  The  second  unit  would  consist  of  digitally- 
controlled  functional  mock-ups  of  several  drawers,  each 
related  to  a  family  of  equipment. 

PREVIOUS  PAGE 
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focus  on  the  teaching  of  tasks  rather  than  on  physical 
realism  of  the  simulator,  and  that  departures  from  maximum 
fidelity  are  often  desirable  for  two  main  reasons:  (1) 
Realism  and  therefore  cost  can  be  reduced  without  impairing 
learning  of  the  tasks;  (2)  addition  of  instructional 
features  which  may  degrade  fidelity  can  greatly  enhance 
training. 

The  author  concludes  "optimal  simulator  design  decisions 
should  be  based  upon  experimental  evidence  gathered  from 
transfer  of  training  studies.  Once  the  relationship  between 
training  on  a  simulator  and  actual  operational  performance 
are  understood,  then  knowledgeable  decisions  about  simulator 
design  can  be  made." 
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as  training  devices  are  cited  from  several  sources;  these 
are  summarized  as:  (1)  Provisions  are  made  for  active 
student  participation;  (2)  there  can  be  repeated  practice 
on  instructional  segments;  (3)  instruction  can  be  paced  to 
the  needs  of  the  individual  student;  (4)  the  accuracy  of 
student  response  can  be  confirmed;  (5)  the  advantages  of 
malfunction/emergency  practice;  (6)  training  equipment 
availability . 

The  issue  of  fidelity  of  simulation  is  discussed  in  respect 
to  several  views  of  fidelity  dimensions  and  the  basis  for 
determining  degree  of  fidelity  required.  A  large  number  of 
empirical  studies  are  cited  in  which  inexpensive,  low- 
fidelity  trainers  were  found  at  least  as  effective  in 
producing  transfer  of  skills  as  far  more  expensive,  high- 
fidelity  counterparts.  Nearly  all  these  studies  were 
confined  to  the  domain  of  procedural  tasks,  however. 

Planning  for  simulation  should  begin  with  a  task  analysis, 
of  which  several  methods  are  available.  Tasks  can  be 
classified  according  to  various  taxonomic  schemes;  the  one 
advanced  here,  per  Gagne  and  Bolles  (1969),  lists  five  chief 
categories:  Procedures,  Motor  Skills,  Indentif ications, 

Conceptual  Tasks,  and  Team  Functions. 

Stage  of  learning  has  a  large  impact  on  selection  of 
training  media,  the  use  of  simulation,  and  fidelity  of 
simulation;  the  section  dealing  with  this  topic  is  the 
longest  in  the  study.  A  general  conclusion  is,  that  the 
needed  degree  of  fidelity  rises  as  training  progresses,  and 
levels  off  at  some  point  at  a  high  level  of  skill. 

The  hypothetical  relationship  between  fidelity,  cost,  and 
transfer  of  training  proposed  by  Robert  B.  Miller  in  1954 
stands  virtually  unchallenged  since  "little  hard  scientific 
evidence  has  been  collected  to  support  or  refute  it."  The 
Miller  model  postulates  that  transfer  increases  as  fidelity 
(and  therefore  cost)  increases,  such  that,  for  any 
particular  stage  of  training,  at  first  transfer  rises 
rapidly  while  fidelity  and  cost  rise  slowly,  then  as  higher 
fidelity  is  attained,  cost  rises  rapidly  while  transfer 
rises  slowly;  at  some  "point  of  diminishing  returns," 
optium  cost-  effectiveness  is  reached  beyond  which 
increasing  fidelity  becomes  overcostly. 

Principles  of  training  are  categorized  according  to  three 
stages  of  learning  (First,  Middle,  and  High).  These 
principles  emphasize  primarily  how  the  simulator  is  used-- 
e.g.,  repitition,  feedback  techniques,  graduated  loading, 
etc.  The  study  ends  with  a  section  of  "Transition 
Training,"  which  propounds  simulator  design  principles  from 
several  sources.  These  sources  agree  that  design  should 
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1.  Author:  Miller,  Gary  G. 

2.  Title:  Some  Considerations  in  the  Design  and 
Utilization  of  Simulators  for  Technical  Training 

3.  Source:  Technical  Report:  AFHRL-TR-7 4-6 5 ,  August  1974, 
NT IS  AD/A-001  630 

4.  Topic  Keywords:  Simulators  ;  Low  Fidelity  ; 

Training  Devices  ;  Maintenance  Training  ; 

Cost-Effectiveness  ;  Training  Effectiveness  ; 

Task  Analysis  . 

5.  Short  Summary:  This  paper  reviews  the  current  technical 
literature  on  simulations  in  training.  It  propounds  rules 
and  principles  for  cost-effective  application  of  simulation, 
emphasizing  the  use  of  simple  low-fidelity  devices  where 
appropriate,  particularly  in  training  procedural  tasks. 

6.  Devices: 

a.  SCI  Console  for  Nike  Hercules 

b.  E-4  Fire  Control  System 

c.  Tank  Hull  Trainer 

d.  Simulated  Maintenance  Task  Environment 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  Technical  Training 
Division,  Air  Force  Human  Resources  Laboratory,  Lowry 
AFB,  CO  80230 

8.  Type  of  article:  Review 

9.  Abstract:  This  study  reviews  technical  literature  from 
1954  to  1974,  with  a  view  to  extracting  general  principles 
for  the  design  and  use  of  simulators  in  technical  training 
(by  which  term  the  author  means  primarily  maintenance 
training) . 

Definitions  are  discussed  in  which  distinctions  are  made 
between  "training  aids"  and  "training  equipment,"  and 
between  "simulators"  and  "training  devices."  Two  systems  of 
classifying  simulators  by  type  are  described. 

Reasons  for  choosing  simulators  over  operational  equipment 


9.  Abstract:  The  purpose  of  this  review  was  "to  analyze 
the  situation  on  the  cost  and  training  effectiveness  of 
training  devices"  which  was  current  at  the  time  of  the  paper 
(1972)  . 

A  review  was  conducted  of  studies  providing  empirical  data 
gathered  in  controlled  experiments,  and  a  summary  made  based 
on  a  commonality  of  findings.  An  appendix  tabulating  19 
"Early  History"  and  13  "Recent"  studies  identifies  what 
equipment  was  simulated,  what  devices  were  studied,  what 
skills  were  taught,  and  briefly  describes  the  experiment  and 
results.  Both  flying  and  non-flying  tasks  are  included. 

The  weight  of  evidence  appears  to  support  the  use  of 
simulators. 

The  experience  of  commercial  airlines  in  adopting  simulators 
for  pilot  training  is  adduced  in  favor  of  the  use  of 
simulators;  a  commercial  airlines  representative  is  cited 
as  saying  that  "for  the  reason  of  cost,  the  airlines' 
objective  is  to  perform  100%  of  training  in  simulators." 
Also,  a  representative  of  the  USAF  Human  Research  Laboratory 
is  cited  as  saying  that  a  100%  goal  is  "meaningful  and 
viable."  The  author  concludes,  "as  a  result  of  this 
demonstration  by  the  airlines  of  the  feasibility  and 
practicality  of  substitution  of  flight  time  by  practice  in 
training  devices,  the  military  should  boldly  adopt  the 
policy  of  substitution  for  the  appropriate  portions  of  the 
flight  training  and  other  operational  training  syllabi." 

A  discussion  of  the  need  for  fidelity  concludes  that 
"training  effectiveness  is  more  a  matter  of  the  manner  in 
which  the  trainer  is  used  than  of  the  fidelity  of  the 
trainer."  The  author  recommends  minimizing  simulator 
fidelity  wherever  possible  in  the  interests  of  saving  costs. 

Roscoe's  (1971)  Transfer  Effective  Ratio  and  Caro's  (1970) 
Equipment-Device  Task  Commonality  Analysis  are  succinctly 
described  in  this  paper. 
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1.  Author:  Micheli,  Gene  S. 

2.  Title:  Analysis  of  the  Transfer  of  Training, 
Substitution  and  Fidelity  of  Simulation  of  Training 
Equipment 

3.  source:  NAVTRAEQUIPCEN  TAEG  Report  2,  1972;  NTIS  No. 
AD-748  594 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Fidelity  of  Simulation  ;  Training  Media  Selection  ; 

Training  Effectiveness  Prediction  . 

5.  Short  Summary:  This  paper  reviews  many  empirical 
studies  of  the  effectiveness  of  training  devices  and 
summarizes  the  commonality  of  findings.  It  concludes  that 
training  effectiveness  depends  more  on  how  the  trainer  is 
used  than  on  the  fidelity  of  the  trainer.  It  recommends  the 
increased  use  of  simulators  for  pilot  training,  not  with  the 
goal  of  exactly  duplicating  operational  equipment,  but  with 
the  goal  of  providing  trainees  with  an  opportunity  to 
practice  behaviors  critical  for  their  subsequent  performance 
on  operational  equipment. 

6.  Devices: 

Link  AN-T-18  ;  12BK-1  Primary  Landing  Trainer  ;  C-3 

Cycloramic  Link  Trainer  ; 

SJN  Cycloramic  (General)  Link  l-CA-2  ;  PBM-OFT  ; 
PB4Y-0FT  ;  3-A-2  and  3-A-35  Aerial  Gunnery  Trainers  ; 

3-A-40  MK  18  Coordination  Trainer  ; 

3-E-7  Ranging,  Tracking  Aiming  Point  Assessor  ; 

3-A-40b  ;  SNJ  OFT  ;  NavBIT  ;  Aetna  Dr ivo- trainer  ; 
Device  1BZ2  Maneuvering  Tactics  Trainer  ; 

15E18  Tactical  ECM  Trainer  ;  GAT-1  ; 

X14A2  Surface  Ship  ASW  Attack  Trainer  ;  2F66A 

Weapons  System  Trainer  ; 

21B20A  Advanced  Submerged  Control  Trainer  ; 

3A105  Tracked  Vehicle  Driving  Trainer  ;  Carrier 
Air  Traffic  Control  Center  Trainer  ;  2F69A  (P-3A 

Weapon  System  Trainer)  ; 

2F90  (TA-4J  Operational  Flight  Trainer)  ;  Whirlymite 

7.  Institutions: 

a.  Sponsor:  Training  Analysis  and  Evaluation  Group, 
Naval  Training  Equipment  Center,  Orlando,  FL  32813 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 
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training  is  based  on  two  misconceptions:  (1)  failure  to 
appreciate  that  all  training  is  a  compromise;  (2)  failure 
to  understand  that  "the  controlling  factor  is  not  physical 
congruence  but  the  (sometimes  subtle)  cues  that  guide  the 
behavior  involved  in  the  performance  of  a  task." 

Proper,  through  task  analysis  procedures  are  vital  to  the 
design  of  effective  synthetic  trainers. 

R&D  projects  in  Naval  maintenance  training  are  classified  by 
the  dimension  of  their  “basic  orientation  or  thrust".  One 
group  of  projects  emphasizes  hardware;  another  emphasizes 
the  development  of  integrated  hardware/software/courseware; 
another  focusses  on  a  general  purpose  "universal" 
maintenance  simulator--examples  of  the  last  are  the 
Behavioral  Technology  Laboratory  ("Rigney")  system,  and  the 
Modular  Integration  of  Training  Information  by  a  Performance 
Aiding  Computer  system. 

Research  models  for  single-rate  and  multi-rate  maintenance 
training — the  Hagan  Automatic  Boiler  Control  and  Woodward 
Electronics  Governor  systems--are  discussed;  one  conclusion 
from  this  research  is  that  "it  is  apparent  that  the  one 
functional  skill  for  which  hi-fidelity  simulated  practice  is 
required  is  fault  isolation." 

A  graduated  learning  sequence,  represented  by  diagram,  is 
described  in  a  four-step  schema:  Knowledge  Acquisition, 
Skill  Acquisition,  Integration,  and  Application,  leading  to 
On-the-Job  training.  The  required  fidelity  of  devices 
during  this  sequence  progresses  in  general  from  low  to 
high--  beginning  with  2-dimensional  displays  and  ending  with 
operational  equipment. 

Future  research  plans  are  discussed  at  the  end  of  the  paper: 
one  goal  is  to  improve  technology  of  instructional  delivery, 
in  particular  overcoming  weaknesses  of  the  self-  paced 
instructional  mode. 


1.  Author:  Malec,  V.  M. 

2.  Title:  untitled 

3.  Source:  unpublished;  NPRDC  1980 

4.  Topic  Keywords:  Maintenance  Training  Simulation  ; 
Behavioral  Technology  Laboratory  System  ;• 

Modular  Integration  of  Training  Information  ; 

Performance  Aiding  Computer  ;  General  Purpose  Simulator  ; 
Fault  Identification  Simulator  ; 

Unitary  Instructional  Systems  . 

5.  Short  Summary:  Several  programs  of  research  and 
development  in  training  of  Naval  maintenance  personnel  are 
discussed.  An  emphasis  is  placed  on  the  advantages  of 
simulator  over  actual  equipment  trainers,  particularly  in 
the  early  stages  of  skill  training. 

6.  Devices: 

a.  Hagan  Automatic  Boiler  Control 

b.  Woodward  Electric  Governor 

c.  A7-HUD 

d.  Simulated  Avionic  Maintenance  Trainer 

e.  Intermediate  Hands-on  Maintenance  Simulator 

7.  Institutions: 

a.  Naval  Personnel  Research  and  Development  Center, 

San  Diego,  CA  92152 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 

9.  Abstract:  This  study  reviews  ongoing  Naval  Research  and 
Development  projects  in  maintenance  training,  with  an 
emphasis  on  the  use  of  simulation  rather  than  actual 
equipment  devices  for  training,  especially  at  the  earlier 
stages  of  skill  aquisition. 

Practical  drawbacks  cited  in  the  use  of  actual  equipment  for 
training  are:  (1)  use  of  obsolescent  equipment:  (2) 
curtailment  of  practice  for  safety  consideration;  (3)  high 
percentage  of  down-time. 


Resistance  to  the  application  of  simulators  in  maintenance 


degrade  transfer  of  training,  although  this  may  only  be 
temporary."  "There  does  not  appear  to  be  a  strict 
requirement  for  high  environment  fidelity  where  easily 
perceived  and  interpreted  information  from  the  real  world  is 
provided  on  visual  displays  in  the  operational  situation." 

The  required  level  of  fidelity  depends  to  some  extent  on  the 
stage  of  learning.  "Early  in  the  training  program 
(procedures  training),  the  trainee  cannot  benefit  from  high 
degrees  of  either  type  of  fidelity.  However,  as  skill  is 
acquired  (familiarization  training),  there  are  requirements 
for  increases  in  both  environment  and  equipment  fidelity 

ll 

•  «  t  • 

Artifical  knowledge  of  results,  or  feedback,  can  be  used  to 
advantage  when  stimulus  conditions  in  the  operational 
environment  are  clearly  perceived;  in  other  circumstances, 
performance  in  the  transfer  environment  may  suffer 
dramatically  when  artifical  knowledge  of  results  has  been 
provided  in  training  and  is  then  withdrawn. 

Designers  should  closely  consider  how  the  interaction 
between  student  and  instructor  is  to  take  place,  and  avoid 
saddling  the  instructor  with  unreasonable  demands. 


briefly  discussed.  'The  bulk  of  this  chapter,  however, 
discusses  training  equipment,  which  is  distinguished  from 
training  aids  by  the  provision  of  active  practice  for  the 
trainee. 

Advantages  and  disadvantages  of  part-task  vs.  whole-task 
trainers  are  discussed.  A  simulator  is  defined  as  a 
part-task  or  whole-task  trainer  which  (a)  attempts  to 
duplicate  the  essential  features  of  a  task  situation,  and 
(b)  provides  for  direct  practice.  Simulator  use  is  demanded 
when : 

(1)  It  is  less  expensive  than  the  actual  equipment  but 
still  represents  the  essential  task  elements; 

(2)  it  is  the  only  feasible  way  to  practice  a  task; 

(3)  it  is  more  reliable  for  practice  purposes  than  the 
actual  equipment; 

(4)  it  permits  more  effective  control  over  the  learning 
process  than  actual  equipment. 

Simulator  use  is  usually  appropriate  in  familiarization, 
skill,  and  transition  training,  but  is  less  often 
appropriate  for  indoctrination  or  procedural  training--in 
these  initial  phases,  unsophisticated  mock-ups  having  very 
low  fidelity  are  usually  adequate. 

General  principles  of  training  device  design  are  briefly 
described.  The  best  overall  design  guideline  for  level  of 
fidelity  is  "that  (higher)  fidelity  of  simulation  is 
requried  when  the  trainee  must  learn  to  make  difficult 
discriminations  between  simulus  events,  and  where  the 
responses  are  either  difficult  to  make  or  are  highly 
critical  to  system  operation." 

Fidelity  can  be  separated  into  three  components: 

(a)  Equipment  fidelity,  the  degree  to  which  the 
simulator  duplicates  the  appearance  and  "feel"  of  the 
operational  equipment; 

(b)  Environmental  fidelity,  the  degree  to  which  the 
simulator  duplicates  the  sensory  stimulation  (excluding 
control  feel)  which  is  received  from  the  task 
situation ; 

(c)  Psychological  fidelity,  the  degree  to  which  the 
simulator  is  perceived  by  the  trainee  as  being  a 
duplicate  of  the  operational  equipment  and  the  task 
situation. 

Psychological  fidelity  should  be  "achieved  on  the  basis  of 
perceptual  limitations  rather  than  on  the  basis  of 
experimental  limitations. " 

In  regard  to  environmental  fidelity,  "gross  deviations  from 
the  display  of  environmental  factors  may  substantially 
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1.  Authors:  Kinkade,  Robert  G.,  &  Wheaton,  George  R. 

2.  Title:  Training  Device  Design 

3.  Source:  Ch.  14  of  Harold  P.  VanCoff  &  Robert  G. 
Kincade  (eds.),  Human  Engineering  Guide  to  Equipment  Design, 
Washington,  DC,  American  Institutes  for  Research,  1972,  pp. 
667-699 

4.  Topic  Keywords:  Whole-task  trainers  ; 

Part-task  trainers  ;  training  aids  ;  equipment  fidelity  ; 
environment  fidelity  ;  psychological  fidelity  ; 
scoring  systems  . 

5.  Short  Summary:  Design  recommendations  for  training 
devices  are  giveh.  The  recommendations  place  the  device 
within  the  context  of  the  training  system  as  a  whole,  and 
deal  with  concepts  of  fidelity  and  other  design  features 
which  influence  effectiveness  of  learning  and  efficiency  f 
instruction. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  American  Institutes  for  Research 

b.  Performing  organization:  American  Institutes  for 
Research,  Washington,  DC  Office 

8.  Type  of  Article:  Conceptual 

9.  Abstract:  This  chapter  from  VanCoff  and  Kinkade's  Human 
Engineering  Guide  to  Equipment  Design  (1972)  places  the 
design  of  training  devices  in  the  context  of  the  training 
system  as  a  whole,  and  discusses  concepts  of  fidelity  and 
other  design  features  which  influence  effectiveness  of 
student  learning  and  efficiency  of  instruction. 

Factors  within  the  overall  training  system  which  bear  on 
training  device  design  are:  training  objectives,  student 
population,  practice  procedures,  the  training  program,  other 
training  devices,  and  evaluation  requirements.  The  approach 
to  design  of  the  training  device,  rather  than  occurring  in 
isolation,  should  weigh  these  factors  heavily.  For  example, 
if  the  training  objective  does  not  include  the  acquisition 
of  a  high  level  of  skill,  the  device  probably  need  not 
provide  high  degrees  of  fidelity. 

Training  devices  may  be  divided  into  two  major  classes: 
training  aids  and  training  equipment.  Training  aids,  used 
by  the  instructor  to  help  present  subject  matter,  are 
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24  months  of  the  inception  of  the  programs. 

Three  principal  reasons  for  success  of  the  project  are 
described  in  the  report: 

(a)  It  was  the  product  of  systematic  application  of 
instructional  systems  design  methodology  which 
integrated  the  synthetic  training  objectives,  academic 
training  objectives,  and  flight  training  objectives. 

The  main  features  of  the  new  programs  included:  (i) 
expanding  the  role  of  the  instructor  pilot  to  that  of  a 
training  manager;  (ii)  individualized  instruction 
through  proficiency  advancement  techniques;  (iii)  crew 
or  team  training;  (iv)  functional  context  training; 
and  (v)  redesigning  academic  instruction...  to 
emphasize  how  to  operate  the  system  rather  than  how  the 
system  operates. 

(b)  The  Coast  Guard  employed  an  exemplary  simulator 
procurement  management  system.  A  Project  Officer, 
assigned  to  the  project  early,  was  kept  at  the  same 
position  throughout  the  procurement  process.  "All 
decisions  made  during  the...  process  were  oriented 
toward  assuring  that  the  simulators  would  be  effective 
training  devices,  since  the  man  making  those  decisions 
knew  he  was  to  be  the  user." 

(c)  "The  project  was  much  a  cooperative  venture  between 
a  military  training  agency  and  a  training  research 
contractor.  It  is  HumRRO's  view  that  it  worked  with, 
rather  than  for,  the  Coast  Guard." 

The  text  also  briefly  describes  the  simulator,  designated 
the  Variable  Cockpit  Training  System.  These  devices  proved 
effective  training  for  the  programs'  objectives  although 
they  lacked  an  exterior  visual  simulation  capability. 
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c.  Providing  the  Student  with  a  Criterion  of 
Performance 

d.  Evaluating  Student  Performance 

e.  Communicating  Evaluations  and  Directions  to 
Students 

f.  Adjusting  Degree  of  Task  Difficulty 

g.  Adjusting  to  Individual  Differences 

h.  Effecting  Transition  of  Students  to  Self-Dependence 

i.  Operating  the  Training  Device 

j.  Concluding  Comments 

Section  III.  The  Trainer  as  a  Demonstrator  of 
Pr inciples 

a.  Functions  of  Demonstration  in  Training 

b.  Selecting  the  Working  Components  of  the 
Demonstration  Trainer 

c.  Constructing  the  Trainer  for  Practicality 

a.  Concluding  Comment 

Section  IV.  The  Use  of  Knowledge  of  Results 

a.  Introduction 

t.  Assumptions  and  Definitions 

c.  Informative  Knowledge  of  Results:  An  Amplification 

a.  Operational  versus  Artificial  Knowledge  of  Results 

e.  Training  the  Student  in  the  Use  of  Learning 
Feedback 

f.  Interaction  of  Learning  Feedback  Information  and 
the  Student's  Hypothesis  of  Cue  or  Response  Relevancy 

g.  Time  Values  in  Learning  Feedback 

h .  Changes  in  Scoring  Tolerences  at  Different  Stages 
of  Learning 

i.  Varying  Emphasis  on  Student's  Error  Comoonents 
].  Changes  in  the  Student's  Selection  and  Use  of 
Feedback 

Section  V.  The  Problem  of  Simulation 

a.  Introduction 

b.  Some  Problems  in  the  Mar.- Mac  nine  Error  System 

c.  Stage  of  Learning  and  Deg: oe  of  Simulation 

Section  VI.  The  Problem  of  Motivation 

a.  Definitions  and  General  Statements 

b.  Social  Group  Differences  m  Incentives  and 
Atti  fades 

c.  Incentives  m  the  Training  Situation 

ti.  Competition  as  a  Motivational.  Influence 
e.-l.  (Other  topics  on  motivation1 

Section  VII.  Preparing  the  Specifications  for  a 
Training  Device 

a.  Introduction 

b.  Stages  Preliminary  to  Design  Recommendations 
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c.  Preparation  of  Specific  Recommendations 

Throughout,  the  book  emphasizes  the  training  of  complex 
perceptual-psychomotor  tasks.  The  author  cites  a  lack  of  a 
"theory  of  learning  which  could  be  extended  to  cover  the 
performance  of  highly  complex  skills  by  intelligent,  symbol¬ 
using  operators."  The  presentation  is  largely  theoretical, 
bolstered  at  points  by  concrete,  homely  examples  rather  than 
by  evidence  from  empirical  studies  (of  which  there  were  few 
available  at  the  time  with  much  validity).  The  author 
states  that  "these  propositions  must  be  offered  as  'best 
guesses'  in  the  making  of  decisions  in  training.  It  is 
hoped  that  psychologists  will  generally  agree  that  they  are 
'best  guesses.'  It  is  also  hoped  that  these  guesses  may  be 
supported,  modified,  or  refuted  by  certainties  as  research 
progresses.  " 

The  handbook's  organization  reflects  three  major  problems  in 
training:  (1)  "how  to  present...  knowledge  of  results  so 

as  to  .  aximize  the  benefits  the  student  can  get  from  the 
responses  he  has  made  in  practice;"  (2)  "the  problem  of 
psychological  simulation  of  tasks  so  that  what  is  learned  in 
one  task  environment  will  have  positive  transfer  effects  to 
another  task  environment;"  (3)  motivation. 

A  major  portion  of  the  book  (110  pages)  is  devoted  to  the 
use  of  Knowledge  of  Results,  which  discusses  learning  as  a 
problem  m  information  processing.  The  65-page  section  on 
"The  Problem  of  Simulation"  focusses  primarily  on  the 
variation  in  simulation  fidelity  requirements  (the  terms 
"realism"  or  "similarity"  are  used  in  this  book)  as  a 
function  of  stage  of  learning.  The  last  section,  on 
"Preparing  the  Specifications  for  a  Training  Device," 
highlights  a  proposed  method  based  on  task  analysis. 

A  number  of  ma]or  points  raised  in  the  book  relevant  to 
transfer  of  training  and  the  fidelity  issue  in  training 
device  design  and  use  may  be  summarized  as  follows: 

(a)  Psychological  similarity,  not  engineering 
similarity  is  the  key  to  effective  transfer  of 
training. 

(b)  Transfer  of  learning  from  one  environment  to 
another  depends  on  the  context  of  the  new  task  environ¬ 
ment  as  well  as  specific  stimuli.  (This  pcint  is  made 
in  reference  to  transition  training,  but  it  applies  to 
transfer  from  simulator  to  operational  environment  as 
well.  ) 

(c)  The  requirements  for  engineering  fidelity  increase 
as  the  stage  of  learning  progresses. 

(d)  "Overlearning"  should  not  be  used  pejoratively. 
"Overlearning"  a  task  to  the  point  of  eliminating 
medication  between  stimulus  and  response  can  be 


extremely  valuable. 

(e)  Physical  differences  between  simulator  and 
operational  environment  weigh  heavily  in  instructor  and 
student  motivation.  Management  of  the  training  program 
should  work  to  instill  positive  attitudes  toward 
training  devices. 

(f)  "Decisions  about  phychological  similarity  or 
dissimilarity  should  not  be  based  upon  the  'analytical 
attitude'  with  respect  to  the  physical  stimulus.  Other 
things  being  equal,  proof  of  the  similarity  of  two 
tasks,  insofar  as  transfer  value  is  concerned,  has  to 
be  based  on  the  transfer  of  training  which  can  be 
obtained. " 

(g)  As  engineering  fidelity  increases,  transfer  of 
training  (dependent  on  stage  of  learning)  generally 
increases,  up  to  a  point  of  diminishing  returns  where 
very  costly  increases  in  engineering  fidelity 
contribute  little  or  nothing  to  training  effective¬ 
ness.  A  graph  used  to  illustrate  this  point  has  been 
subsequently  employed  by  many  other  researchers  in  the 
field. 
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1.  Author:  Montague,  William  E. 

2.  Title:  Is  Simulation  Fidelity  the  Question? 

3.  Source:  L'.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  547, 
"Research  Issues  in  the  Determination  of  Simulator  Fidelity: 
Proceedings  of  the  ARI  Sponsored  Workshop  23-24  July  1981," 
November  1981;  Robert  T.  Hays,  editor,  pp.  93-109 

4.  Topic  Keywords:  Simulation  Fidelity  ;  Mental  Models  ; 
Transparency  ;  Analogical  Correspondence  . 

5.  Short  Summary:  The  requirement  for  physical  and 
stimulus  fidelity  is  subordinate  to  the  requirement  for  task 
fidelity,  in  the  quest  for  training  effectiveness. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  C.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  5001  Eisenhower  Avenue., 
Alexandria,  VA  22333 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  The  foundation  of  the  author's  approach  is 
revealed  in  the  following  statements:  "What  is  required  for 
training  simulators/devices  is  that  they  must  provide  the 
cues,  the  opportunities  for  trainees  to  respond,  make  and 
correct  errors,  and  observe  the  consequences  of  their 
actions.  Simulation  that  teaches  well  may  violate  the 
physical  and  temporal  isomorphism,  and  its  adequacy  in 
training  may  be  due  to  that  fact.  This  idea  is 
counterintuitive  to  the  naive  view  that  the  most  important 
design  goal  is  to  provide  high  physical  and  stimulus 

f iael lty  .  " 

This  paper  attempts  to  identify  guidelines  for  designing 
training  devices  that  effectively  teach  the  task,  quite 
apart  from  questions  of  device  fidelity.  The  following 
major  goals  of  training  devices/simulators  are  describee: 

(a)  "to  build  up  in  the  learner's  mind/memory  a 
representation  of  the  tasks  to  be  performed  and  how  the 
system  functions;  the  goal  is  to  build  up  the  trainee's 
understanding  of  the  system..."  (b)  "to  produce  smooth, 
error  free,  execution  of  needed  procedures/responses";  (c) 
"to  teach  m  a  safe,  less  complicated  environment,  and 
perhaps  show  the  consequences  of  unsafe  actions";  (d)  "to 


519 


provide  for  the  development  of  skills";  (e)  "to  provide 
tutor ing. .. that  give  hints,  coaching,  warnings, 
explanations. " 

Essential  to  effective  training  are  the  following 
character istics  of  the  training  environment: 

(a)  The  involvement  of  the  learner 

(b)  An  easily  recognizable  analogical  correspondence 
between  the  simulation  and  the  real  task 

(c)  Opportunities  to  practice  procedures  frequently  and 
to  commit  errors 

(a)  "Transparency",  or  clear  knowledge  of  results. 

None  of  the  foregoing  characteristics  necessarily  require 
high  physical  or  stimulus  fidelity. 

Two  principal  concepts  are  central  to  a  view  of  training 
which  makes  fidelity  a  secondary  goal  in  designing  a 
simulator  and  training  program.  First  is  that  of  the 
internal  "mental  model"  which  is  used  by  the  learner  to 
understand  the  operational  reality  which  is  being  simulated. 
This  understanding  might  in  many  instances  be  brought  about 
without  a  high  degree  of  realism  in  simulation.  The  second 
concept  is  the  necessity  for  task  fidelity.  A  major  problem 
in  determining  task  fidelity  is  the  lack  of  well-developed 
task  analysis  procedures,  especially  for  complex  tasks. 

Naval  sonar  operator  training  devices  and  programs  are  used 
to  illustrate  some  of  these  points. 


1.  Author:  Montelermo,  Melvin  D. 

2.  Title:  Training  Device  Design:  The 
Simulation/Stimulation  Controversy 

3.  Source:  NAVTRAEQU I PCEN  Technical  Report  IH-287,  July 
1977 

4.  Topic  Keywords:  Media  Selection  Models  ; 

Training  Devices  ;  Training  Effectiveness  ; 

Pierside  Training  . 

5.  Short  Summary:  This  study  develops  a  prototype  model  of 
the  training  device  selection  process.  The  model  is  based 
on  seven  factors  pertinent  to  device  effectiveness:  cost, 
reliability,  maintainability,  safety,  facility  requirements, 
trailing  features,  and  modifiability. 

6.  Devices:  Many  devices  are  mentioned  in  review 

7.  Institutions: 

a.  Sponsor:  Commanding  Officer,  Naval  Training 

9)  •  Equipment  Center,  Orlando,  FL  32813 

b.  Performing  organization:  Human  Factors  Laboratory, 
Naval  Training  Equipment  Center,  Orlando,  FL  32813 

8.  Type  of  Article:  Methodological 

9.  Abstract:  This  study  sets  forward  what  the  author  terms 
a  "primitive  model  of  the  training  device  selection 
process.  "  The  model  is  constructed  to  guide  choices  between 
device  options,  which  have  in  the  past  been  guided  largely 
by  tradition. 

It  is  noted  that  traditional  selection  policies  in  many 
branches  of  the  service  have  favored  actual  equipment 
training  devices,  particularly  in  maintenance  training,  but 
that  the  acceptance  of  simulators  has  been  gaining  ground. 
Contradictory  assertations  from  either  side  of  the  actual 
equipment  versus  simulator  controversy  are  cited.  The 
report  proceeds  to  discuss,  in  reference  to  these  options 
and  with  numerous  specific  examples,  seven  factors  which 
constitute  the  proposed  selection  model:  (1)  reliability; 

(2)  maintainability;  (3)  safety;  (4;  cost;  (5)  facility 
requirements;  (6)  training  features;  and  (7) 
modi f iabil i ty . 

The  author  contends  that  consideration  of  these  factors 
"should  enable  an  experienced  training  technician  to  select 
and  design  traning  devices  which  are  more  responsive  to 
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training  system  requirements. " 

Whenever  possible,  the  selection  should  be  made  by  a  team 
whose  members  combine  expertise  in  instructional  technology, 
engineering  technology,  the  opertional  equipment  to  be 
taught,  and  procurement  procedures.  The  team's  goals  should 
be  to : 

a.  involve  the  instructors  who  will  be  using  the 
device ; 

b.  develop  a  training  program  that  capitalizes  on  the 
device's  particular  advantages; 

c.  develop  a  training  program  for  instructors; 

d.  develop  a  training  program  to  move  the  trainees 

from  the  device  to  the  actual  equipment; 

e.  ensure  that  the  device  is  completely  "debugged" 
before  putting  it  into  use; 

f.  prepare  a  maintenance  program  for  the  device; 

g.  prepare  a  modification  program  to  ensure  that  the 

device  is  always  kept  up  to  date; 

h.  ensure  that  all  team  programs  are  adquately 
documented  prior  to  disbanding. 

The  report  also  discusses  recent  developments  in  low- 
fidelity  maintenance  training  simulators,  though  without 
discussing  the  fidelity  issue  per  se . 

In  addition,  a  third  training  option,  that  of  using  actual 
equipment  as  a  trainer,  not  in  the  schoolhouse  but  in  a 
restricted  field  environment  such  as  a  ship  at  pierside-- 
"Pierside  Training "--i s  discussed  with  ten  examples. 

The  author  concludes  by  recommending  further  work  to  expand 
and  refine  the  training  media  selection  model;  this  work 
should  focus  on  (1)  other  factors;  (2)  interaction  among 
the  factors;  (3)  relative  importance  of  the  factors;  and 
(4)  circumstances  which  affect  the  potency  of  each  factor. 


1.  Author:  National  Technical  Information  -Service 

2.  Title:  Simulators  in  Education  and  Training,  March 
1 980- J  une  1982 

3.  Source:  U.S.  Department  of  Commerce  National  Technical 
Information  Service  PB82-810953,  August  1982;  supersedes 
PB81-806622  and  PB80-807787 

4.  Topic  Keywords:  Simulators  ;  Operations  Training  . 

5.  Short  Summary:  This  bibliography  references  and 
abstracts  279  reports  and  journal  articles  in  research  on 
simulation  and  simulation  technology,  with  the  exception  of 
flight  simulators  and  flight  simulation. 

6.  Devices:  Various  devices  cited  in  279  references 

7 .  Institutions : 

a.  Sponsor:  National  Technical  Information  Service, 
Springfield,  VA 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review  (Bibliography) 

9.  Abstract:  The  citations  in  this  bibliography  cover 
research  on  simulators  and  simulation  technology  in  a  broad 
range  of  educational  and  training  settings.  The  earliest 
reference  cited  is  April  1946  (a  patent  filing);  the  latest 
reference  cited  is  from  March  1982,  with  the  great  majority 
falling  in  the  years  1979-1980.  References  are  to  studies 
of  simulation  in  game  playing  and  computer  models,  as  well 
as  military  and  commercial  training. 

Citations  contain  the  minimum  following  information,  where 
available:  Title,  Author (s).  Corporate  Author (s), 

Sponsoring  Agency,  Pages  in  Report,  Report  Date,  Report 
Number,  NTIS  Subject  Category,  Keywords,  Abstract  of 
Content,  NTIS  Order  Number,  Microfiche  Price  Code,  and  Paper 
Copy  Price  Code. 

Abstract  length  varies  up  to  approximately  200  word  (most 
are  100-150  words ) . 

Approximately  18  of  the  references  are  patent  filings  for 
simulation  devices.  Approximately  14  deal  with  games.  A 
breakdown  by  rough  subject  categories  is  as  follows: 
Maintenance  Training  and  Maintenance  Training  Devices:  25 
citations;  Operations  Training  (including  War  Games)  and 
Operations  Training  Devices:  64  citations;  Miscellaneous 
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Authors:  Orlansky,  J. ,  &  String,  J. 


2.  Title:  Cost-Effectiveness  of  Maintenance  Simulators  for 
Military  Training 

3.  Source:  IDA  Paper  P-1568,  August  1981 

4.  Topic  Keywords:  Maintenance  Simulation  ; 
Cost-Effectiveness  ;  Fidelity  of  Simulation  . 

5.  Short  Summary:  A  review  of  research  on  maintenance 
training  by  simulation,  accompanied  by  .a  cost-analysis  of 
simulators,  concludes,  with  some  qualifications,  that 
simulators  are  in  general  more  cost-effective  than  actual 
equipment  trainers. 

6 .  Devices : 

a.  Generalized  Sonar  Maintenance  Trainer 

b.  EC  II  ,  EC  II  VLP 

c.  Automated  Electronics  Maintenance  Trainer 

d.  Generalized  Maintenance  Training  System 

e.  Fault  Identification  Simulator 
(Hagen  Automatic  Boiler) 

f.  6883  Convertor/Flight  Control  Systems  Test  Station 

g.  MA-3  Test  Stand 

h.  Trident  Integrated  Radio  Room 

i.  Trident  High  Pressure  Air  Compressor 

j.  Trident  Air  Conditioner 

7 .  Institutions : 

a.  Sponsor:  Office  of  the  Under  Secretary  of  Defense 
for  Research  and  Engineering,  The  Pentagon,  Washington, 
DC  20301 

b.  Performing  Organization: 

Institute  for  Defense  Analysis,  400  Army-Navy  Drive, 
Arlington,  VA  22202 

B.  Type  of  Article:  Review 

3.  Abstract:  This  study  evaluates  the  cost-effectiveness 
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of  maintenance  simulators  versus  actual  equipment  devices 
for  training  military  maintenance  technicians.  The  general 
conclusion,  stated  with  slight  qual if ica tion,  is  that 
simulators  are  more  cost-effective  than  actual  equipment  for 
training.  This  conclusion  is  based  on  the  following 
evidence  gleaned  from  a  review  of  empirical  studies  and  a 
cost  analysis: 

a.  Training  Effectiveness,  established  via  empirical 
studies : 

(1)  In  12  studies  conducted  since  1967,  student 
achievement  in  12  courses  that  used  maintenance 
simulators  was  the  same  as  or  better  than  that  in 
comparable  courses  that  used  actual  equipment 
trainers;  in  one  case,  student  achievement  with  a 
maintenance  simulator  was  less. 

(2)  In  the  one  case  where  on-the-job  performance 
was  evaluated,  supervisors'  ratings  showed  no 
difference  between  students  trained  with  the 
simulator  or  an  actual  equipment  trainer. 

(3)  In  three  cases  where  such  data  were  collected, 
time  savings  (with  the  simulator)  were  22,  50,  and 
50  percent. 

(4)  Most  students  who  use  maintenance  simulators 

have  favorable  attitudes  toward  their  use;  9 

instructors  are  split  about  equally  in  having 
favorable,  neutral,  or  negative  attitudes  toward 
the  use  of  these  simulators. 

b.  Cost  Analysis  of  Equipment,  including  6  standard 
simulator  systems  or  models,  17  non-standard  systems, 
and  one  Computer-Aided-Instr uction  system,  found: 

(1)  The  cost  to  design,  develop,  and  fabricate  one 
unit  of  a  simulator  is  less  than  60  percent  of  the 
unit  cost  of  its  counterpart  actual  equipment 
trainer  in  7  cases  out  of  a  sample  of  11;  in  the 
remaining  four  cases  the  simulator  cost  more  than 
the  actual  equipment  trainers. 

(2)  Once  developed,  the  cost  of  fabricating  an 
additional  unit  of  a  simulator  is  less  than  20 
percent  of  the  unit  cost  of  its  counterpart  actual 
equipment  trainer  in  9  of  those  11  cases;  in  only 
one  case  did  the  simulator  cost  more  to  fabricate 
than  the  actual  equipment  trainer. 

(3)  In  tiie  one  available  case  of  a  life  cycle 
cost-  effectiveness  evaluation;  the  simulator  was 
as  effective  is  the  actual  equipment  trainers  both 
at  schooi  and  on  the  job.  The  total  costs  for  the 
same  stuc ent  ^oad  over  a  15  year  period  were 
estimatec  at  between  $1.5  million  for  the 
simulator  and  $3.9  million  for  the  actual 
equipment  trainer. 

The  conclusion  as  to  cost-effectiveness  of  simulators  is 
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qualified  by  three  considerations:  a  limited  number  of 
valid  comparisons  are  available;  effectiveness  has  been 
measured  by  school  rather  than  on-the-job  performance  in  all 
but  one  case;  and  the  cost  analysis  is  based  primarily  on 
acquisition  rather  than  life-cycle  costs. 

Additional  research  is  required  to  clairfy  several  points. 
Especially,  there  is  a  need  to  determine  how  training 
methods  affect  later  job  performance.  Furthermore,  research 
could  turn  up  more  ways  of  increasing  cost-effectiveness, 
particularly  by  modifying  simulator  fidelity:  "the 
interrelationships  of  complexity,  fidelity,  and  cost  of 
training  equipment  and  the  transfer  of  training  from 
training  devices  to  operational  equipment  clearly  deserve 
systematic  attention. " 


1.  Author:  Payne,  Thomas  A. 

2.  Title:  Conducting  Studies  of  Transfer  of  Learning:  A 
Practical  Guide 

3.  Source:  AFHRL-TR-81-25 ,  January  1982 

4.  Topic  Keywords:  Transfer  of  training  ; 

Transfer  of  learning  ;  Transfer  effectiveness  ratio  ; 
Transfer  performance  measurement  ;  Flying  training  . 

5.  .Short  Summary:  An  outline  of  the  methods  of  conducting 
a  meaningful  transfer  of  learning  study  makes  a  number  of 
specific  recommendations  and  cites  examples  from  previous 
experiments,  as  well  as  constructing  a  general  sequential 
framework  consisting  of  eleven  steps  to  be  followed  by 
researchers . 

6.  Devices:  no  specific  devices 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
( AFSC) ,  Brooks  AFB ,  TX  78235 

b.  Performing  Organization:  University  of  Dayton 
Research  Institute,  Dayton,  OH  45469 

8.  Typc  of  Article:  Methodological 

9.  Abstract:  This  paper  constitutes  a  guiae  primarily 
for  conducting  studies  of  transfer  of  learning  from 
flight  simulators  to  actual  aircraft.  A  distinction  is 
made  between  the  term  "transfer  of  learning"  and  the 
term  "transfer  of  training"  which  is  commonly  equated 
with  it,  in  order  to  clarify  that  it  is  the  effect  of 
training,  not  the  process  itself,  that  is  the  object  of 
jtu dy  . 

Two  methoos  for  evaluation  of  transfer  are  presented, 
followed  by  eleven  steps  designed  to  furnish  valid 
results  for  those  models. 

The  two  evaluative  models  are:  (1)  Percent  Transfer  of 

Learning,  and  \2)  Transfer  Effectiveness  Ratio.  The 
first  expresses,  as  a  porcentace,  the  amount  of 
real-aircraft  training  time  that  is  saved  by  prior 
training  in  a  simulator,  or  the  graded  performance 
increment  that  results  from  prior  training  in  a 
simulator.  The  Transfer  Effectiveness  Ratio  expresses 
the  real  aircraft  training  time  saved  as  a  fraction  of 
the  amount  of  prior  simulator  training  time.  The 
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latter  model  has  the  advantage  of  directly  comparing 
the  effectiveness  of  aircraft  time  with  simulator  time, 
thus  offering  a  formula  for  computing 
cost-effectiveness. 

The  eleven  steps  identified  as  critical  in  conducting  a 
transfer  of  learning  study  are  as  follows: 

(1)  Definition  of  the  immediate  problem--for  example, 
the  transitioning  of  pilots  to  a  new  type  of  aircraft 

(2)  Definition  of  the  task 

(a)  Phase  of  the  curriculum 

(b)  Specific  sub-tasks 

(3)  Selection  of  subjects 

(a)  Population  to  be  represented 

(b)  Matching  versus  random  assignment  to  groups 

(4)  Performance  Measurement  Technique 

(a)  Meaningful,  logical  relationship  between 
measures  and  tasks 

(b)  Establishment  of  performance  criterion 

(c)  Trials  to  criterion  measure 

(d)  Performance  grading  measures;  "subjective" 
and  "objective" 

(e)  Practical  significance 

(f)  Recording  techniques 

(g)  Automated  systems 

(5)  Instructors 

(a.)  Instructors  train,  not  the  device 

(b)  instructor  as  researcher 

(c)  Instructor  role  in  identifying  simulator- to- 
aircraft  differences 

(d)  Adherence  to  study  design 

(e)  No  instruction  during  measurement 

(f)  Balance  of  instructors  between  groups, 
simulator  and  aircraft 

(6)  Planning  for  sufficient  study  time 

(7)  Avoidance  of  dilutant  t actor s- -e . g .  ,  time  delays 
between  blocks  of  training  and  transfer 

(8)  Scheduling  in  advance 

(9)  Operational  Training  Environment 

(10)  Testing  the  study  method  before  taking  final  data 

(11)  Analysis  of  results 

(a)  Interpretation  of  findings  not  to  rest  solely 
on  probability  (p)  levels. 

(b)  Inspect  raw  performance  data  first 

{c  Cross-comparisons  of  statistical  test  results 
with  raw  data 

(a)  Endue  concern  with  underlying  assumptions  of 
parametric  tests. 

Throughout  this  paper,  heavy  emphasis  is  placed  on  proper, 
thorough  planning  prior  to  the  study,  to  include  a 
preliminary  study  of  possible  experimental  confounds,  and 
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correctable  equipment  deficiencies-  Another  continuing 
concern  of  the  researchers  must  be  the  adequacy  of 
resources. 


1.  Author:  Pitrella,  F.  D. 


2.  Title:  Problems  and  Methods  in  Evaluating  Force 
Steering  and  Other  Flight  Control  Modes:  A  Study  and  an 
Experimental  Design 

3.  Source:  Forschungsinstitut  fur  Anthropotechnik  Bericht 
Nr.  20  (Research  Institute  for  Human  Engineering  Report  No. 
20),  December  1974 

4.  Topic  Keywords:  Force  steering  ;  Control  wheel  ; 

Flight  control  modes  ;  Instrument  flight  simulation  ; 
Research  simulation  ;  Fidelity  . 

5.  Short  Summary:  Theoretical  considerations  and  an 
experimental  design  are  proposed  for  study  of  the  flight- 
performance  effectiveness  of  a  partly  automated  aircraft 
flight  control  system,  termed  Force  Steering,  in  a 
fixed-base  instrument  flight  simulator. 

6.  Device:  Instrument  flight  simulator  with  medium-high 
physical  fidelity  and  medium-high  functional  fidelity 

7.  Institutions: 

a.  Sponsor:  Forschungsinstitut  fur  Anthropotechnik, 
Luf telberger  Strasse  L  1123,  5309  Meckenheim,  Germany 

b.  Performing  organization:  same 

8.  Type  of  Article:  Theoretical/Methodological 

9.  Abstract:  Force  steering,  a  flight  control  system  which 
seeks  to  optimize  the  interaction  between  the  human  pilot 
and  the  automated  pilot,  had  theoretical  advantages  over  the 
manual  and  automatic  piloting  systems  in  use  at  the  time  of 
this  study,  but  these  had  not  been  experimentally  proved. 

The  objective  of  the  research  proposed  in  this  paper  was  to 
study  the  effectiveness  of  Force  Steering  in  instrument 
approach  and  landing  of  the  aircraft,  in  comparison  with  (1) 
fully  automated  approach  control;  (2)  semi-automatic 
control;  and  (3)  fully  manual  control. 

Force  Steering  is  a  compound  control  mode  which  includes 
three  types  of  control  signals  in  various  combinations 
depending  on  pilot  decision.  It  enables  the  pilot  to 
participate  in  the  system  at  different  levels  of  the 
aircraft  control  hierarchy,  rather  than  being  compelled  to 
choose  between  submitting  to  the  Flight  Director,  or  taking 
over  complete  control.  Fundamental  to  the  Force  Steering 
design  is  the  concept  that  machines  are  more  suitable  for 
performing  lower-level  functions  such  as  changing  the  rudder 


to  maintain  a  selected  glidepath,  while  humans  are  better  at 
higher  level  functions  such  as  determining  the  glidepath. 

It  was  proposed  to  evaluate  the  Force  Steering  model  by 
determining  the  relative  contributions  of  the  four  modes. 
This  was  to  be  done  in  a  fixed-base  simulator  in  three  ways: 

(1)  by  directly  comparing  landing  approach  performing 
differences  among  the  four  control  modes;  (2)  by  measuring 
and  recording  the  percentage  of  time  that  Force  Steering  is 
characterized  predominately  by  each  of  the  contributory 
modes;  (3)  by  measuring  and  recording  the  percentage  of 
time  that  a  pilot  makes  control  inputs  and  the  force 
magnitude  of  those  inputs. 

In  order  to  discriminate  between  the  influences  of  status 
displays  and  Flight  Director  displays  on  the  pilot,  the 
experiment  was  to  use  three  different  display 
configurations:  (1)  conventional  panel  without  the  display 

of  Flight  Director  information  (information  restricted  to 
status  error  displays) ;  (2)  conventional  panel  with  both 

Flight  Director  and  status  error  information  displayed  (3) 
conventional  panel  with  Flight  Director  information  but 
without  status  error  information. 

Experimental  hypotheses  were  advanced  on  the  following:  (1) 
normal  condition  landing;  (2)  turbulent  condition  landing; 
(3)  automaticity  levels  during  differing  conditions. 

Discription  of  Proposed  Experiment: 

a.  Number  of  groups:  3  or  4 

b.  Description  of  groups:  to  be  selected  from  12-18 
minimum,  pilots  with  small  or  moderate-size  jet 
experience  or  training  with  instrument  landing  approach 

c.  Test  or  trials/timing:  in-process  trials 

d.  Number  of  different  types  of  measures:  2 

e.  Description  of  measurements  and  ratings:  objective 
quantitative  measurements  of  (1)  flightpath  errors, 
deviations;  and  (2)  control  inputs 

f.  Experimental  setting/training  context:  Laboratory, 
hands-on 

g.  Statistical  methods:  3x3  factorial  ANOVA;  Tukey's 
test  of  nonadditivity 

h.  Variables  being  manipulated: 

(1)  Training  device:  as  above,  section  6 

(a)  Force  steering  modes 

(b)  Fully  automatic  modes 

(c)  Semi-automatic  mode 

(d)  Fully  manual  mode 

(2)  Fidelity  Levels: 

(a)  Physical:  Medium-high 

(b)  Functional:  Medium-high 

(3)  Type  of  task/skill  required:  operations, 
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cognitive,  psychomotor,  perceptual,  part-task 

(4)  Task  difficulty:  High 

i.  Stage  of  training:  skill  (experimental  only) 

j.  Trainee  sophistication:  intermediate 

k.  Incorporation  of  device  into  P.O.I.:  lock-step 

l.  User  acceptance  or  attitude:  not  applicable 

m.  Use  of  instructional  features: 

(1)  Intensity:  assumed  will  be  intensive 

(2)  Features  used:  not  specified 

The  degree  and  fidelity  of  simulator  realism  required  to 
produce  valid  results  was  discussed.  It  was  proposed  that  a 
submaximal  degree  of  realism  could  achieve  the  desired  level 
of  validity,  because  the  research  goals  were  not  to 
demonstrate  precisely  what  would  happen  in  the  real  world, 
but  rather  to  obtain  a  relative  rank  order  of  performance 
among  the  four  control  modes  under  study.  In  no  case  was 
the  principle  of  realism  to  interfere  with  the  requirements 
of  achieving  good  experimental  control. 

Other  issues  discussed  were:  the  performance  measurement 
problem;  the  question  of  task  difficulty;  the  selection  of 
an  adaptive  variable  as  the  primary  dependent  variable; 
assymetrical  transfer  of  training;  experimenter  bias  and 
control  technique  bais;  performance  and  adaptive 
measurement  instability  with  angular  information;  adequacy 
of  data  sampling  and  matching;  and  other  experimental 
control  issues. 


1.  Authors:  Purifoy,  George  R.  Jr.,  &  Benson,  Eugene  W. 

2.  Title:  Maintenance  Training  Simulators  Design  and 

Acquisition:  Summary  of  Current  Procedures 

3.  Source:  AFHRL-TR-7 9-23 ,  November  1979 


4.  Topic  Keywords:  Maintenance  Training  Simulation  ; 
Instructional  Systems  Development  ; 

Computer-assisted  instruction  . 

5.  Short  Summary:  This  paper  describes  administrative 
procedures  involved  in  the  design  and  acquisition  of 
maintenance  training  simulators.  Both  the  Instructional 
Systems  Development  (ISD)  procedures  and  the  system  Program 
Office  (SPO)  procedures  are  discussed. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory 
(AFSC) ,  Brooks  AFB,  TX  78235 

b.  Performing  Organization: 

Applied  Science  Associates,  Inc. ,  Box  158,  Valencia,  PA 
16059 

8.  Type  of  article:  Theoretical 

9.  Abstract:  This  paper  is  the  first  in  a  series  exploring 
the  problems  of  maintenance  training  simulator  design  and 
acquisition.  It  focusses  on  the  administrative  procedures 
in  the  decision-making  process  and  has  only  cursory 
relevance  to  the  training  issues  that  are  involved  in  the 
design  of  devices  and  programs  of  instruction.  It  does 
illustrate,  however,  how  the  training  issues  are  embedded  in 
the  large  organizational  activity  of  device  development  and 
acquisition. 


Th  design  and  procurement  activity  is  broken  down  into  three 
major  phases:  (1)  identification  of  requirements;  (2) 
development  of  specifications;  and  (3)  procurement.  A 
design  process  model  is  described  to  assist  in  the 
identification  of  requirements  and  development  of 
specifications.  The  purpose  of  this  design  model  is  to  lay 
the  groundwork  for  forthcoming  hierarchical  and  associative 
relationships  between  information  about  tasks  and 
appropriate  training  equipment  characteristics  and  training 
applications  by  comparing  it  to  currently  used  procedure. 


The  decision  sequence  of  the  model  is  as  follows: 
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(1)  Determine  job-relevant  skills  and  knowledge; 

(2)  Specify  skills  and  knowledge  which  must  be  learned; 

(3)  Identify  skills  and  knowledge  best  learned  on 
equipment; 

(4)  Group  skills  and  knowledge  by  training  equipment 
class; 

(5)  Specify  how  well  training  equipment  related  skills 
and  knowledge  must  be  learned; 

(6)  Sequence  all  training  requirements; 

(7)  Establish  training  equipment  design  concepts; 

(8)  Development  a  preliminary  plan  of  instruction; 

(9)  Finalize  functional  equipment  characteristics; 

(10)  Document  training  equipment  design. 

A  key  to  the  selection  of  design  features  and  instructional 
program  is  the  concept  that  for  any  set  of  tasks  to  be 
trained,  there  are  a  number  of  alternative  combinations  of 
training  media  which  can  be  employed  to  successfully  achieve 
required  learning. 


1.  Authors:  Rolfe,  J.  M. ,  &  Waag,  Wayne  L. 

2.  Title:  Fidelity  of  Flight  Simulation  for  Pilot  Training 

3.  Source:  Royal  Air  Force  Education  Bulletin  No.  18, 
Autumn  1980 

% 

4.  Topic  Keywords:  Fidelity  of  Simulation  ; 

Flight  Simulation  ;  Cost-Effectiveness  Ratio  . 

5.  Short  Summary:  Alternatives  to  designing  training 
devices  for  .maximum  fidelity  are  discussed,  with  a  focus  on 
cost-effective  means  for  accomplishing  specific  training 
obj  ectives. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Royal  Air  Force  Support  Command,  Upwood, 
England 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  study  is  based  on  the  premise  that  cost- 
effective  training  by  simulation  is  seldom  achieved  through 
the  uncritical  adoption  of  the  "fidelity  model"  of  simulator 
design  which  assumes  that  the  more  realism  in  the  simulator, 
the  better  for  training.  Three  practical  considerations 
underlie  this  premise:  (1)  the  high  cost  of  achieving  high 
simulator  fidelity;  (2)  the  fact  that  the  ultimate  in 
realism  is  unattainable  in  a  simulator  by  definition;  and 
(3)  research  data  indicate  that  effective  training  can  be 
achieved  without  high  fidelity  of  simulation,  in  many  cases. 

The  study  outlines,  with  the  help  of  a  block  diagram, 
methods  for  selecting  the  training  media,  and  determining 
the  fidelity  requirements  for  those  media,  within  the 
context  of  the  training  system  as  a  whole.  The  key  to 
selection  of  appropriate  training  device  design  features  is 
the  clear  specification  of  training  objectives,  which  pivots 
on  a  mission  analysis  of  the  constellation  of  tasks  to  be 
learned  by  the  aircrew. 

Principles  of  simulator  training  which  have  been  established 
by  research  are: 

(1)  "The  flight  simulator  is  an  effective  means  of 
teaching  and  maintaining  aircrew  skills." 

(2)  "Substantial  amounts  of  time  in  the  air  can  be 
replaced  by  the  use  of  simulators." 


(3)  "The  ease  with  which  tasks  can  be  learned...  and 
transferred...  varies  with  the  nature  of  the  task.... 
Procedural  skills  will  transfer  readily  but  will  be 
forgotten  unless  practiced  regularly.  Perceptual  motor 
skills  transfer  less  completely...." 

(4)  "The  effects  of  simulator  training  are  most 
apparent  in  the  critical  period  immediately  after  the 
student  moves...  to  aircraft." 

(5)  "Training  effectiveness  is  not  determined  solely  by 
the  appropriateness  of  the  training  device  to  the  . . . 
task.  How  the  device  is  used  can  influence  its 

.  effectiveness  to  an  equal  or  greater  extent." 

(6)  "Training  effectiveness  can  be  improved  by  reducing 
the  level  of  complexity  in  the  training  device.... 

(Also) . . .  changes  to  the  equipment  may  be  necessary  in 
order  to  facilitate  feedback...  and  to  allow  the 
manipulation  of  the  training  task." 

(7)  "Motivation  is  a  key  element  in  achieving  effective 
training  in  a  ground  trainer." 

(8)  "The  degree  of  realism  of  simulation  required  will 
depend  on  the  nature  of  the  task,  the  level  of 
experience  of  the  students,  and  the  level  of  (expcted) 
proficiency. ..." 

(9)  "Visual  tasks  learned  in  the  simulator  show 
positive  transfer  to  the  aircraft....  Few  studies  have 
been  accomplished  for  tasks  other  than  transition...." 

(10)  "There  is  insufficient  research  data  to  make 
conclusive  statements  regarding  the  value  of  motion  in 
the  training  context.  " 

The  authors  conclude  that  "the  research  data  base  regarding 
the  requirements  for  effective  training  is  quite  limited." 

A  Cost  Effectiveness  Ratio,  which  is  derived  from  dividing 
the  Transfer  Effectiveness  Ratio  by  the  Training  Cost  Ratio, 
is  discussed  and  illustrated  by  two  examples  from  Royal  Air 
Force  flight  training  programs.  The  Transfer  of  Training 
model  for  evaluating  simulator  training  effectiveness  is 
discussed  and  its  limitations  pointed  out--particularly  the 
inconstancy,  in  most  experiments,  of  how  the  simulator  is 
used. 
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1.  Authors:  Roscoe,  Stanley  N.  , 

2.  Title:  Review  of  Flight  Training  Technology 

3.  Source:  U.S.  A.R.I.  Research  Problem  Review  76-3, 

July  1976 

4.  Topic  Keywords:  Helicopter  Flight  Simulation  ; 
Nap-of-the-Earth  Aircrew  Training  ;  Residual  Attention  ; 
Automatically  Adaptive  Training  ; 

Computer-Assisted  Instruction  . 

5.  Snort  Summary:  A  survey  of  possible  options  for 
simulator  designs  appropriate  for  Nap-of-the-Earch 
helicopter  flight  training  suggests  part-task  simulated 
training,  using  cinematic  methods,  combined  with  air 
training,  as  the  most  effective  solution. 

6.  Devices:  2-B-24  Synthetic  Flight  Training  System 
(mentioned) 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Alexandria,  VA  22333 

b.  Performing  Organization:  Anacapa  Sciences, 
Incorporated, 

University  of  Illinois  Institute  of  Aviation 

8.  Type  of  Article:  Review 

9.  Abstract:  The  purpose  of  this  report  was  to  review  the 
state  of  the  art  of  aircrew  training  technology, 
particularly  in  simulation,  as  part  of  a  program  to  identify 
areas  in  which  nap-of-the-earth  (NOE)  aircrew  training  might 
be  most  readily  improved. 

The  principal  limitation  in  applying  simulation  to  training 
NOE  helicopter  f 1 ight--which  involves  below-treetop-  level 
maneuvers — is  the  extreme  comlexity  of  the  visual 
environment.  Flight  simulators  at  the  time  of  the  report 
could  not  provide  closed-loop  capacity  for  this  kind  of 
flight--that  is,  in  a  way  where  the  pilot  could  control'  the 
simulated  motion  of  the  aircraft.  The  alternative 
recommended  is  the  combination  of  cinematic  simulation--the 
use  of  films  taken  in  real  flight--with  air  training.  Open- 
loop  film  presentations,  which  can  permit  closed-loop 
control  of  pitch,  yaw,  roll,  and  speed,  but  not  of  the  three 
translational  degrees  of  freedom,  have  been  used 
successfully  in  training  high-speed,  low  altitude  navigation 
and  have  promise  for  NOE  flight  training. 
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A  number  of  other  topics  are  discussed  in  addition  to  that 
of  outside-cockpit  visual  simulation.  Heavy  stress  is 
placed  on  attaining  the  highest  possible  procedural 
fidelity.  Procedural  skills  are  more  aircraft-specific  and 
more  easily  forgotten  than  perceptual-motor  skills.  Other 
topics  which  are  discussed  come  under  the  general  heading  of 
"innovations  in  flight  training":  Automatically  Adaptive 
Training;  Computer-Assisted  Instruction,  Adaptive 
Measurement  of  Residual  Attention,  Automatic  Performance 
Measurement,  and  Interactive  Computer-Control-Display 
Devices.  Of  these,  only  Computer-Assisted  Instruction  (CAI) 
and  Adaptive  Measurement  of  Residual  Attention  are  strongly 
recommended  for  the  NOE  flight  appliction.  Standardization 
of  instruction  and  individually  paced  instruction  are 
advantages  of  CAI.  Measurement  of  Residual  Attention  serves 
to  discriminate  among  ability  of  pilots  and  to  assess  their 
readiness  to  cope  with  the  stress  of  combat  or  equipment 
malfunction. 

A  concise  discussion  of  automated  performance  measurement 
emphasizes  it  is  not  instrumentation  which  is  the  problem, 
but  rather  detemining  what  characteristics  to  measure,  and 
what  kind  of  parameters  furnish  the  most  useful  data. 


1.  Author:  Semple,  Clarence  A. 
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2.  Title:  Guidelines  for  Implementing  Training 
Effectiveness  Evaluations 

3.  Source:  Technical  Report,  NAVTRAEQUIPCEN  72-C-0209-3; 
NTIS  AD-778  349,  April  1974 

4.  Topic  Keywords:  Training  Effectiveness  Evaluation  ; 
Human  Factors  . 

5.  Short  Summary:  This  paper  presents  guidelines  for 
planning,  implementing,  and  documenting  training 
effectiveness  evaluations. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization: 

Manned  Systems  Sciences  Inc. ,  8949  Reseda  Blvd. ,  Suite 
206,  Northridge,  CA  91324 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  The  training  effectiveness  evaluation  is 
employed  to  determine  the  effects  on  training  of  a 
modification  in  an  ongoing  program.  The  rationale  for 
performing  evaluations  is  that  "training  personnel  need  such 
estimates  to  be  able  to  project  skill  and  knowledge  levels 
which  will  result  from  modified  training,  determine  cost 
effectiveness  of  training,  husband  all  training  resources, 
and  identify  areas  where  further  modifications  may  be 
desirable."  This  report  deals  with  three  major 
considerations  in  evaluation: 

(1)  The  evaluation  process  should  have  the  least 
possible  impact  on  the  program  under  evaluation,  both 
to  avoid  altering  the  training  itself,  and  to  earn  the 
good  will  and  cooperation  of  training  personnel. 

(2)  There  are  alternative  models  of  evaluation  of  which 
the  transfer  of  training  experiment  is  only  one. 
Selection  among  the  alternatives  constitutes  a  major 
initial  planning  step. 

(3)  The  evaluation  must  be  geared  not  merely  to  obtain 
reliable  meaningful,  and  practical  information,  but  to 
obtain  reliable,  meaningful,  and  practical  information. 

Concerning  impact  on  the  training  program,  it  is  vital  that 
the  evaluation  personnel  adopt  a  guest  relationship  to  the 
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host  training  program  personnel.  The  evaluation  should  be 
planned  and  conducted  on  a  "not  to  interefere"  basis. 

Concerning  choices  of  evaluative  model  or  models,  it  must  be 
understood  that  "there  are  alternative  models  which  are 
applicable  to  training  effectiveness  evaluations...  The 
further  one  regresses  from  the  classic  transfer  of  training 
model,  the  less  concrete,  defensible  and  generalizable  the 
resulting  information  becomes."  Four  levels  of  evaluation 
are  outlined  in  a  table  for  quick  reference. 

Obtaining  reliable,  meaningful  and  practical  information 
depends  not  only  on  techniques  of  information  gathering  but 
upon  orienting  the  evaluation  to  clearly  specified  goals  of 
the  training  program.  "Achieving  agreed  upon  definitions  of 
why  training  modifications  are  being  made...  is  a 
cornerstone  of  a  useful  training  effectiveness  evaluation." 

General  recommendations  for  dealing  with  management 
problems — especially  lead-time  planning  problems--and 
specific  measurement,  analysis,  and  documentation  techniques 
are  discussed. 
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1.  Author:  Semple,  Clarence  A. 

2.  Title:  Executive  Summary:  A  Report  of  the  Simulator 
Training  Requirements  and  Effectiveness  Study  (STRES) 

3.  Source:  Technical  Report,  AFHRL-TR-80-63 ,  July  1980; 
NTIS  No.  AD-A094  381 

4.  Topic  Keywords:  Flight  simulation  ;  Life  Cycle  cost  ; 
Aircrew  training  devices  ;  Training  effectiveness  ; 

Cost  Modeling  ;  Fidelity  ;  Instructional  Support  Features 
;  Worth  of  Ownership  ;  Transfer  of-  Training  . 

5.  Short  Summary:  This  review  summarizes  the  contents  of 
six  reports  on  aircrew  training  devices  performed  as  part  of 
the  Simulator  Training  Requirements  and  Effectiveness  Study 
(STRES),  and  indexes  these  technical  reports. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Air  Force  Human  Resources  Laboratory  , 

0  Advanced  Systems  Division,  Wr ight-Patterson  AFB ,  OH 

45433 

b.  Performing  Organization: 

Canyon  Research  Group,  Inc.  ,  741  Lakefield  Rd. ,  Suite 
B,  Westlake  Village,  CA  91361 

8.  Type  of  Article:  Review 

9.  Abstract:  The  purposes  of  this  report  were  (1)  to 
summarize  the  content  of  six  other  technical  reports 
prepared  during  the  Simulator  Training  Requirements  and 
Effectiveness  Study  (STRES),  and  (2)  to  provide  a  master 
index  to  the  content  of  program  technical  reports  dealing 
with  the  design,  use,  life  cycle  cost  and  worth  of  ownership 
of  aircrew  training  devices.  The  basic  goals  of  the  STRES 
study  include:  (1)  to  develop  criteria  to  match  training 
requirements  to  simulator  features;  (2)  to  define 
principles  for  effective  use  of  aircrew  training  devices; 

(3)  to  develop  principles  for  matching  instructional 
features  to  training  requirements;  and  (4)  to  accumulate 
data  on  cost-effectiveness  of  aircrew  training  devices. 

The  following  issues  are  addressed  in  the  studies  which  are 
summarized:  aircrew  training  device  fidelity; 

instructional  support  features,  utilization  of  training 
devices  in  aircrew  training  programs;  training  device  life 
cycle  costs;  and  worth  of  ownership. 
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1.  Author:  Spangenberg,  Ronald  W. 

2.  Title:  Selection  of  Simulation  as  an  Instructional 
Medium 

3.  Source:  First  International  Learning  Technology  Society 
&  Exposition  on  Applied  Learning  Technology,  Society  for 
Applied  Learning  Technology,  21-23  July  1965,  Proceedings 
Vol.  IV:  "Future  of  Simulators  in  Skills  Training",  pp. 

6  5. 

4.  Topic  Keywords:  Simulation  ; 

Instructional  Media  Selection  . 

5.  Short  Summary:  This  paper  briefly  describes  seven 
advantages  of  simulation  as  a  training  medium  and  advocates 
the  case  study  approach  as  a  means  of  establishing  a 
predictive  body  of  knowledge  about  simulator  training 
effectiveness  and,  ultimately,  improving  planning  and  design 
of  simulations. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Society  for  Applied  Learning  Technology 

b.  Performing 

Organization: Air  Force  Human  Resources  Laboratory 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  paper  briefly  describes  the  advantages 
of  simulation  as  a  training  medium  and  advocates  the  case 
study  approach  as  a  means  to  improving  the  planning  and 
design  of  simulations. 

In  the  selection  of  simulation  as  a  training  medium,  it 
should  be  borne  in  mind  that  tasks  with  "abstract 
information  inputs  requiring  the  production  of  abstractions 
are  less  likely  to  require  that  the  training  task 
environment  resemble  the  actual  task  envi ronment ”--i . e . , 
simulators  of  high  realism  are  more  appropriate  to  the 
learninq  of  concrete  tasks. 

Seven  unique  advantages  of  simulation  are  cited: 

(a)  Can  provide  extra  knowledge  of  results  (feedback) 

(b)  Can  increase  the  number  and  frequency  of  problems 
to  be  practiced 

(c)  Can  compress  time 

(a)  Can  vary  the  sequence  of  tasks 

(e)  Can  provide  guidance  and  stimulus  support 


from  maximum  fidelity  (e.g.,  display  enhancement,  augmented 
feedback)  in  order  to  increase  the  instructional  efficiency 
of  the  device.  Flexibility  is  also  a  key  issue,  in  that 
design  options  must  be  capable  of  handling  potential 
revision,  or  addition  of  training  objectives. 

Determination  of  fidelity  rests  on  two  considerations:  what 
or  how  much  to  simulate,  and  how  realistically  to  simulate 
it.  "In  principle,  the  design  criterion  is  clear:  it 
provides  the  minimum  fidelity  of  simulation  needed  to 
develop  and  exercise  the  human  performances  required  in  the 
operational  environment."  Four  basic  levels  of  fidelity  are 
identified:  (1)  High-Fidelity  Representation;  (2) 

Deliberate  Reduction  in  Fidelity  Where  the  Engineering 
State-of-the-Art  is  Adequate;  (3)  Reduction  in  Fidelity 
Tolerances  when  the  Engineering  State-of-the-Art  is  Less 
than  Adequate;  (4)  Deliberate  Departures  from  Realism. 

Deliberate  reductions  in  fidelity  can  be  assessed  from  a 
negative  standpoint--that  is,  in  the  interests  of  reducing 
costs,  how  much  fidelity  can  be  sacrificed  without 
significantly  reducing  trainee  performance.  Fidelity 
reduction  resulting  from  limitations  in  engineering 
capability  are  more  problematical:  "the  solutions  are  less 
than  desired,  and  design  practices...  rely  more  on 
empirical  determinations  since  precise  quantitative  design 
information  is  lacking."  On  the  other  hand,  deliberate 
departures  from  realism  calculated  to  enhance  training 
effectiveness  reflect  the  incorporation  of  training 
technology  into  design  practices. 


1.  Authors:  Smode,  Alfred  F. ,  &  Hall,  Eugene  R. 

2.  Title:  Translating  Information  Requirements  into 
Training  Device  Fidelity  Requirements 

3.  Source:  Proceedings  of  the  Human  Factors  Society  ,  19th 
Annual  Meeting,  Dallas,  TX,  October  14-16,  1975,  pp.  33-39 

4.  Topic  Keywords:  Simulation  ;  Fidelity  ; 

Training  Effectiveness  ;  Front  end  analysis  . 

5.  Short  Summary:  Observations  on  issues  in  training 
device  fidelity  support  the  view  that  inaccurate  or 
inadequate  information  may  be  equally  at  fault  with 
inadequate  engineering  fidelity  when  equipment  fails  to 
fulfill  training  needs.  A  more  solid  information  base  on 
the  human  factors  involved  in  training  is  necessary  to 
adequately  determine  device  fidelity  requirements. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Training  Analysis  and  Evaluation  Group, 
U.S.  Navy,  Orlando  FL  32813 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 

9.  Abstract:  Flaws  in  training  device  design  are  not  due 
solely  to  lack  of  engineering  fidelity.  In  fact, 
reduced-fidelity  devices  have  been  shown  in  many  cases 
equally  as,  or  more  effective,  than  high-fidelity  devices  in 
training  for  the  same  task.  Just  as  important  as 
engineering  fidelity  to  device  design  is  the  application  of 
human  learning  factors. 

"The  development  of  device  training  requirements...  is  a 
deceptively  complex  process,  not  fully  understood."  Analyses 
of  tasks  to  be  learned  must  include  not  only  the  operational 
performance  itself,  but  also  the  stimulus  configurations 
which  cue  the  action,  and  influential  environmental  factors. 
Selection  of  sub-tasks  to  be  learned  is  a  "complex  iterative 
process  requiring  multiple  trade-offs."  Much  of  the  training 
inadequacies  of  simulators  are  due  to  imprecision  or  errors 
in  information  derived  from  these  analyses. 

Translation  of  training  requirements  into  device  design 
involves  the  specification  of  what  functions  the  equipment 
is  to  perform  and  detailed  design  prescriptions  to  p  oduce 
these  functions.  This  may  involve  deliberate  devi=ti 'ns 


Trainee  Station  Design,  in  relation  to  cost-effectiveness, 
hinges  on  the  questions  of  what  task  elements  are  to 
simulated,  and  what  degree  of  fidelity  is  required  for  those 
elements  selected.  "Of  concern  is  the  extent  to  which 
certain  events  must  be  simulated  and  their  relative 
importance  and  criticality  for  training. " 

Four  levels  of  fidelity  are  identified:  (1)  High  Fidelity 
(engineering  state-of-the-art  is  adequate;  (2)  Fidelity 
achievable  where  reduction  in  tolerances  is  required 
(engineering  state-of-the-art  is  less  than  adequate) ;  (3) 

Simulate  generalized  functions;  and  (4)  deliberate 
departures  from  realism  (qualitative  deviations  from  the 
operational  system  being  simulated).  Deliberate  reductions 
of  fidelity,  or  departures  from  realism,  which  are  not  the 
result  of  engineering  limitations,  have  either  of  two 
distinct  purposes:  either  to  save  costs  without  lowering 
training  effectiveness,  or  to  introduce  non-realistic 
features,  such  as  display  enhancement,  which  increase  the 
training  capability  or  flexibility  of  the  device. 

Instructor  Station  Design.  Two  desiderata  dominate  this 

design  area:  (1)  automating  key  instructor  functions;  and 

(2)  generalized,  modular  construction.  These  two  features  « 

can  increase  flexibility  and  economy  in  design  and  enable 

"substantial  modifications  in  instructional  capability 

within  a  given  device  or  across  similar  devices  without 

radical  change  in  instructor  station  configuration." 

Requirements  for  the  instructor  station  fall  into  two 
principal  domains:  (1)  information  requirements;  and  (2) 
control  requirements.  The  instructor  must  have  means  both 
to  manipulate  the  simulated  conditions  and  to  monitor  and 
evaluate  the  student's  performance.  Fourteen  chapters  in 
this  section  describe  the  requirements  pertinent  to  the 
structure  and  control  of  training  during  off-line,  pre¬ 
mission,  enroute  training,  and  post-exercise  operations. 

A  detailed  list  of  instructor  functions  is  provided,  which 
identifies  for  each  function  its  place  in  the  total  training 
sequence,  and  the  station  capability  and  instructor 
involvement.  An  instructor  functions  flow  analysis  and 
checklist  of  information  requirements  are  also  provided. 

Evaluation  of  Design:  This  section  describes  the  techniques 
for  evaluating  the  design  in  three  principal  steps:  (1) 

Mock-Up  Evaluation;  (2)  Static  and  Part-System  Verification 
Testing;  and  (3)  Instuctional  Suitability  Testing.  The 
role  of  supporting  research  is  also  discussed. 
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1.  Author:  Smode,  Alfred  F. 


2.  Title:  Training  Device  Design:  Human  Factors 
Requirements  in  the  Technical  Approach 

3.  Source:  NAVTRAEQUIPCEN  Technical  Report  71-C-0013-1, 
August  1972 

4.  Topic  Keywords:  Simulation  Fidelity  ;  Human  Factors  ; 
Instructor  Station  ;  Performance  Measures  . 

5.  Short  Summary:  This  document,  primarily  a  handbook  of 
techniques  and  procedures,  presents  guidelines  for  applying 
human  factors  knowledge  to  the  design  of  training  devices, 
and  describes  a  method  for  correlating  instructional 
requirements  with  engineering  design  alternatives. 

6.  Devices: 

a.  A7E  Operational  Flight  Trainer,  Device  2F100 

b.  Synthetic  Flight  Training  System,  Device  2B24 

c.  Device  1D23  Air  Navigation  Trainer 

7.  Institutions: 

a.  Sponsor:  Naval  Training  Equipment  Center,  Orlando, 
FL  32813 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Methodological 

9.  Abstract:  This  document  presents  guidelines  for 
applying  human  factors  knowledge  to  the  design  of  training 
devices.  It  is  primarily  a  handbook  of  techniques  and 
procedures,  and  provides  little  conceptual  information.  The 
techniques  and  procedures  described  assist  the  process  of 
the  "Technical  Approach"  which  links  the  initial 
documentation  and  description  of  training  need,  with  the 
engineering  of  the  device.  Execution  of  these  procedures 
prior  to  the  exploration  of  engineering  options  should  help 
ensure  that  technical  considerations  are  subordinated  to 
human  factors  considerations.  Emphasis  is  on  the 
instructional  capability  of  the  device,  and  nearly 
two-thirds  of  the  material  deals  with  the  design  of  the 
instructor  station. 

Three  major  areas  are  dealt  with  separately:  (1)  Training 
Station  Design;  (2)  Instructor  Station  Design;  and  (3) 
Evaluation  of  Design. 
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definition  of  adaptive  variables  and  difficulties 
in  the  measurement  of  performance. 

(7)  Deliberate  Departure  from  Realism  in  Design. 
This  section  presents  concepts  and  techniques  for 
deliberately  departing  from  realism  in  design  in 
order  to  enhance  training  effectiveness.  Topics 
are:  augmented  information  feedback,  display 

enhancement,  guidance  supports,  and  mediation. 

(a)  Augmenting  information  feedback,  in  terms 
of  rapidity,  variety,  and  supplementation, 
has  been  shown  to  increase  speed  of  learning 
acquisition.  Studies  of  transfer  are  not 
discussed  here. 

(b)  Display  enhancement  provides  for  the 
enhancing  of  signals  to  ensure  detection  and 
identification  in  cluttered  displays. 

(c)  Guidance  support,  or  cue-prompting,  has 
been  shown  useful  in  acquisition  of  part-task 
learning.  The  experimental  evidence  is  still 
rather  slim,  and  ambiguous.  Studies  of 
transfer  are  not  discussed  here. 

(d)  Mediation  involves  selection 
non-equipment  options  as  part  of  the  device. 
This  approach  reduces  or  simplifies  hardware 
and  employs  the  instructor  in  the  training 
loop.  Examples  of  this  technique  are  cited 
without  citing  experimental  evidence. 

d.  Demonstration  of  Principles  in  Design  of  a  Specific 
Trainer:  This  section  illustrates  the  application  of 

the  prescribed  methodology  to  the  design  of  an  actual 
device.  This  reconstruction  is  based  for  the  most  part 
on  analyses  made  by  Dunlap  &  Associates  in  the 
development  of  a  short-based  team  trainer  for  destroyer 
and  cruiser  ASW  operations. 
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lags,  stimulus-response  compatibility  and 
vigilance. 

Design  techniques  for  enhancing  displays  for 
training  are  discussed.  Topics  are:  information 
coding,  and  actual  versus  simulated  targets. 

(5)  Measurement  System  Design.  In  this  section, 
"emphasis  is  placed  on  automatic  human  performance 
monitoring  and  recording  means  for  providing 
measures  and  scores  that  can  be  organized, 
manipulated  and  applied  to  achieve  the  purposes  of 
assessment."  Current  training  devices  are 
generally  deficient  in  well-developed  measurement 
capability.  Instructor  scoring  is  the  norm.  The 
measurement  weakness  results  from  designing  for 
simulator  fidelity  with  little  concern  for 
measurement.  Measures  are  often  not  closely 
linked  with  transfer  proficiency,  and  the 
difficulty  of  measurement  often  dictates  the 
practice  of  measuring  what  is  easily  measurable 
rather  than  what  is  of  practical  significance. 
Little  commonality  exists  between  measurement  for 
training  and  measurement  for  system  test  and 
development.  Measurement  of  tactical 
decision-making  is  especially  problematic  because 
rarely  are  absolute  right  or  wrong  choices  made. 
Also,  the  measurement  of  team  performance  is 
difficult.  Means  for  quantifying  behavioral 
elements  must  be  established  before  consistently 
valid  measurements  can  be  taken. 

(6)  Adaptive  Training  Strategies.  As  considered 
here,  adaptive  training  is  restricted  to  computer- 
controlled  automation  of  the  instructor  function 
in  a  closed-loop  feed-back  system,  where  task 
difficulty  is  progressively  changed  to  match  the 
trainee's  level  of  skill.  Variations  in 
conceptualization  of  adaptive  training  are 
described,  with  the  two  principal  themes  being: 

(a)  assistance  from  the  computer  via  display 
augmentation  is  adjusted  as  a  results  of  the 
subject's  instantaneous  performance;  (b)  an 
average  of  the  subject's  error,  or  "continuously 
distributed  error",  is  compared  with  an  error 
threshhold  to  make  the  system  either  easier  or 
more  difficult. 

No  empirical  evidence  existed  at  the  time  of  this 
report's  publication  to  confirm  the  effectiveness 
of  adaptive  training  techniques  in  simulator 
training.  However,  many  theoretical  advantages 
inhere  in  adaptive  training  which  make  some 
features  of  it  intuitively  desirable. 

The  remainder  of  this  section  explores  problems 
associated  with  the  conceptualization  and 
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precise  in  sensing  movement  as  the  motion  senses. " 
However,  it  is  acknowledged  that  "evidence 
supporting  the  decision  to  incorporate  motion  into 
the  ground-based  trainer  is  yet  incomplete  in 
terms  of  actual  transfer  benefits;  the  majority 
of  the  findings  do  not  yield  clear  expressions  of 
the  extent  of  transfer  of  training." 

The  remainder  of  this  section  covers  priorities  of 
motion  requirements,  according  to  kind  of  motion 
and  task,  sensory  threshholds  for  motion 
perception,  achieving  acceleration  cues  in 
simulation,  acceleration  onset  and  washout, 
scaling  of  motion  forces,  differential  pressures 
on  the  body  as  motion  cues,  shearing  forces,  some 
specifications  for  achieving  motion,  visual  motion 
interaction,  and  vibration. 

(3)  Vehicle  Control  Requirements.  "Assuming  cost 
minimization  as  a  goal,  design  and  fabrication 
costs  may  be  reduced  by  subtracting  unneeded 
simulator  fidelity  from  a  contemplated  design. " 
Topics  covered  in  this  section  are: 

(a)  the  purpose  of  simulator  controls  (as 

distinct  from  aircraft  controls) ; 

(b)  desired  control  fidelity; 

(c)  evaluative  criteria  for  simulator 

controls; 

(d)  research  issues. 

In  regard  to  fidelity,  the  practical  questions, 
both  of  cost  and  of  the  benefits  of  features  which 
lower  fidelity  while  enhancing  training 
versatility  (e.g.,  augmented  feedback),  are 
addressed.  The  following  issues  in  simulation 
fidelity  are  discusses;  motivational  benefits  of 
realism;  fidelity  of  configuration;  requirements 
for  interactive  tasks;  simplification  of 
aerodynamic  equations;  program  cycle  time; 
control’  types,  and  effective  time  constant. 

(4)  Information  Processing.  "Viewing  man  as  a 
single-  channel  limited  transmission  capacity 
information  processing  system  provides  the 
potential  for  specifying  system  components  most 
compatible  with  his  capabilities. " 

Types  of  information  processing  tasks  are 
identified:  conservation  of  information  tasks, 

information  filtering  tasks,  and  information 
condensing  tasks.  The  human  information  handling 
capacity  is  described.  Variables  affecting  the 
perceiving  and  processing  of  information  are 
identified:  detecting  and  discriminating  visual 

signals,  amount  and  distribution  of  input 
information,  distortions  in  input  information,  the 
influence  of  noise,  signal  intermittency ,  time 


training  simulators,  and  field  contexts,  and 
generalizes  the  findings  of  the  issues  in  device 
design.  Seven  topics  are  investigated,  and  summarized 
in  order  below: 

(1)  Visual  Simulation.  Effective  visual 
simulation  in  training  does  not  depend  on 
achieving  a  one-to-one  copy.  The  challenge  is  to 
meet  task  fidelity  requirements. 

Visual  characteristics  are  identified  as  follows: 
size  of  field,  size  of  objects,  luminance  of 
objects  and  background,  shape  of  objects,  color  of 
objects  and  background,  distance  of  objects  in  the 
field,  motion  of  objects,  variations  in 
brightness.  These  characteristics  are  discussed 
in  detail,  with  regard  to  their  role  in 
simulation. 

A  technique  is  described  for  developing  visual 
simulation  specifications.  Advantages  and 
disadvantages  of  various  visual  simulation  media 
are  itemized.  "...  The  most  important  aspect  of 
this  analytic  technique  is  that  it  deals 
explicitly  with  those  factors  which  are  important 
in  the  selection  of  simulation  hardware  but  which 
are  not  recognized  as  quantitative  system 
parameters."  To  illustrate,  the  technique  is 
applied  to  three  candidate  simulation 
technologies — CCTV-model,  wide-angle  open-loop 
movie,  and  computer-based  imagery--as  a  function 
of  the  same  set  of  simulation  requirements. 

(2)  Platform  Motion  Simulation.  This  section 
emphasizes  acceleration  and  deceleration  force 
magnitudes.  A  number  of  issues  of  prime  concern 
in  the  aerospace  environment  are  minimally 
important  to  training  simulator  design,  primarily 
those  dealing  with  physiological  reaction  to 
acceleration  stresses. 

Findings  from  a  number  of  studies  are  adduced  to 
support  the  case  for  the  contribution  of  simulated 
platform  motion  to  improved  student  pilot 
performance  in  the  simulator.  Especially  in  the 
case  of  simulated  flight  under  heavy  turbulence 
conditions,  "motion  aids  in  the  building  of  a 
response  pattern  which  transfers  positively  to  the 
aircraft. 

The  importance  of  motion  cues  appears  to  be  a 
function  of  the  experience  level  of  the  pilot,  but 
whether  motion  in  ths  simulator  is  more  important 
for  initial  skill  acquisition  or  for  training 
pilots  of  some  experience  will  depend  upon  the 
task,  among  other  things. 

The  argument  in  favor  of  motion  is  based  primarily 
on  the  contention  that  "the  visual  sense  is  not  as 


(2)  Design  coordinates  two  distinct  but  closely 
interrelated  functions: 

(a)  trainer  station  design,  which  focuses  on 
achieving  fidelity  of  simulation; 

(b)  the  management  of  training,  focussing  on 
structure,  control,  monitoring,  and  recording 
of  training; 

(3)  Limitations  on  design  options  are: 

(a)  the  methods  for  achieving  human  factors 
design  solutions  are  inexact; 

(b)  the  information  base  of  concepts  and  data 
pertinent  to  design  is  weak; 

(c)  considerable  gaps  in  design  data  exist 
and  some  information  has  marginal  value. 

(4)  International  sacrifices  of  physical  fidelity 
or  operational  realism  can  be  made  in  the  device 
which  either  do  not  detract  from,  or  actually 
increase,  the  training  effectiveness  of  the  device 
(e.g.,  signal  enhancement); 

(5)  The  most  efficient  selection  of  degree  of 
fidelity  should  be  based  on  perceptual  equivalence 
to  the  operational  environment; 

(6)  Long-term  costs  of  device  modification  may 
weigh  against  seeking  initial  cost  savings  which 
inhibit  the  flexibility  of  the  device;  computer 
advances  may  make  it  counter-productive  to  skimp 
on  high  fidelity. 

b.  Methodology:  This  section  describes  the  roughly 
chronological  sequence  of  the  following  steps  in  design 
development : 

(1)  Definition  of  the  purpose  of  the  training 
system; 

(2)  Analysis  of  the  operational  system; 

(3)  Analysis  of  tasks  involving  the  trainee(s): 

(a)  task  structure; 

(b)  training  objectives; 

(c)  identification  of  simulation  elements 
relative  to  own-vehicle  characteristics, 
target  characteristics,  and  the  media. 

(4)  Gross  device  hardware  definition; 

(5)  Definition  of  the  characteristics  of  the 
operational  environment  to  simulate  for  training; 

(6)  Representing  the  operational  environment  in 
the  trainee  compartment (s) ; 

(7)  Provisions  for  the  management  of  training  at 
the  instructor  station  (design  for  the  structure 
and  control  of  training)  . 

Extensive  classification  of  simulation  elements,  and  of 
measures,  are  included  in  this  section. 


c.  Applicable  Concepts  and  Date:  This  section 
examines  the  literature  of  studies  in  the  laboratory, 


1.  Author:  Smode,  Alfred  F. 


2.  Title:  Human  Factors  Inputs  to  the  Training  Device 
Design  Process 

3.  Source:  NAVTRADEVCEN  Technical  Report  69-C-0298-1, 
September  1971 

4.  Topic  Keywords:  Transfer  of  Training  ; 

Measurement  Systems  ;  Augmented  Feedback  ;  Platform  Motion  ; 
Adaptive  Training  ;  Fidelity  ;  Flight  Simulation  ; 

Cue  Prompting  ;  Human  Factors  ;  Training  Effectiveness  . 

5.  Short  Summary:  A  comprehensive  catalog  is  made  of 
methods  and  issues  relevant  to  human  factors  in  the  design 
of  training  devices  (simulators),  with  reference  to 
pertinent  research  and  learning  concepts. 

6.  Devices:  Device  14A2,  ASROC/ASW  Early  Attack  Warning 
System  Trainer  is  discussed  at  length  as  an  example. 

7.  Institutions: 

a.  Sponsor:  Human  Factors  Laboratory,  Naval  Training 
Device  Center,  Orlando,  FL  32813 

b.  Performing  Organization:  Dunlap  and  Associates  , 
Inc.,  Darien,  CT  60820 

8.  Type  of  Article:  Review/Theoretical 

9.  Abstract: 

a.  Introduction:  This  report  presents  guidelines  for 
human  factors  inputs  to  the  design  of  synthetic 
training  systems,  primarily  in  the  area  of  operations 
with  special  attention  to  flight  simulation,  and  is 
addressed  to  those  persons  having  responsibility  for 
human  factors  inputs  during  conception  and  development 
of  the  design  of  training  devices.  It  outlines  a 
chronological  methodology  for  simulator  deverlopment  in 
view  of  purposes,  tasks,  and  equipment;  it  discusses 
human  factors  concepts  and  data  relevant  to  the  design 
process;  and  it  details  the  implementation  of  the 
method  using  the  concrete  example  of  a  complex 
shore-based  team  trainer  (Early  Attack  Warning  System 
Trainer) . 

Key  concepts  set  forth  in  the  introduction  are: 

(1)  The  overriding  consideration  in  training 
system  design  is  the  transfer  of  training  from  the 
simulator  to  the  real-world  environment; 
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little  training  in  the  principles  of  learning; 

(4)  Increasing  dissimilarity  of  new  training  devices  to 
actual  equipment; 

(5)  Instructors'  fear  of  negative  training  on  an 
unrealistic-seeming  device. 

One  prominent  symptom  of  user  neglect  of  Research  and 
Development  information  is  that  specifications  writing  for 
new  devices  relies  chiefly  on  old  models.  Also,  "all  too 
often  in  the  device  specification  process,  too  little 
attention  is  given  to  the  particular  and  specific  training 
tasks  and  training  objectives  that  the  device- is  to  be  used 
to  train. " 


i 
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1;  Author:  Semple,  Clarence  A. 

2.  Title:  Fidelity:  W nat  We  Know  and  Who  Should  Know  It 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Technical  Report  547, 

November  1981:  Research  Issues  in  the  Determination  of 
Simulator  Fidelity:  Proceedings  of  the  ARI  Sponsored 
Workshop,  23-24  July  1981 

4.  Topic  Keywords:  Training  Device  User  ; 

Fidelity  of  Simulation  ;  Negative  Transfer  . 

5.  Short  Summary:  Problems  in  user  acceptance  of 
deliberately  lowered  fidelity  in  training  devices  are 
discussed.  It  is  recommended  those  in  the  R£  community 
consider  how  to  educate  users  to  accept  arguments  in  favor 
of  lowered  fidelity. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  U.S.  A.R.I.  for  the  Behavioral  and 
Social  Sciences,  5001  Eisenhower  Avenue,  Alexandria,  VA 
22333 

b.  Performing  Organization: 

Canyon  Research  Group,  Inc.  741  Lakefield  Road,  Suite 
B,  Westlake  Village,  CA  91361 

8.  Type  of  Article:  Theoretical 

9.  Abstract:  This  brief  paper  deals  chiefly  with  problems 
in  user  acceptance  of  deliberately  lowered  fidelity  in 
training  simulators.  Research  has  found  it  cost-effective 
to  reduce  fidelity  in  some  instances;  however, 
understanding  among  users  of  these  findings  is  very  limited. 

"Users"  are  identified  as  "those  involved  in  writing  device 
specifications,  developing  the  courses  in  which  various 
training  devices  will  be  used,  the  instructors  who  implement 
the  instruction  using  the  device,  and  the  students  who  are 
being  trained. "  The  following  obstactles  to  user  acceptance 
of  research  findings  on  fidelity,  training,  and  device 
design  are  cited: 

(1)  Lack  of  technical  information  on  the  part  of  user; 

(2)  Turnover  and  reassignment  of  (especially  military) 
personnel,  preventing  key  decision-makers  from  reaching 
the  top  of  their  learning  curves  in  the  field; 

(3)  Inadequacy  and  lack  of  structure  in  training  users 
in  device  utilization;  especially,  instructors  have 
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The  summary  of  studies  on  fidelity  covers  the  following 
topics:  visual  system  fidelity,  flight  characteristics 
fidelity,  playform  motion  systems,  force  cuing,  and  visual- 
motion  interaction.  Important  points  under  these  topics 
are : 

(1)  While  visual  systems  in  simulated  flight  training 
appear  to  be  a  must,  available  scientific  and 
operational  information  comes  up  short  on  specific 
guidance  for  optimizing  visual  systems. 

(2)  The  pursuit  of  very  high  flight  characteristics 
fidelity  may  be  excessive;  however,  again  there  is  a 
lack  of  systematic  scientific  information  in  this  area. 

(3)  Research  on  the  training  effectiveness  of  platform 
motion  cuing  is  inconclusive. 

(4)  Force  cuiny  features,  e.g.,  G-seats,  are  new,  and 
research  on  their  training  effectiveness  is  as  yet 
inadequate. 

(5)  The  lack  of  synchronization  of  visual  and  motion 
cues  due  to  limitations  of  simulation  equipment  can 
seriously  impair  training;  little  empirical  data  exist 
on  tolerable  limits  or  desirable  minimums  for 
asynchrony  among  visual,  motion,  and  cockpit  display 
cues. 

The  summary  of  studies  on  instructional  support  features 
covers  the  following  topics:  the  instruct  function,  the 
monitoring  and  evaluation  of  performance,  the  control  and 
individualization  of  training,  the  controller  function, 
instructor/operator  console  design  and  location,  and 
instructor/operator  training. 

The  summary  of  studies  on  utilization  of  aircrew  training 
devices  covers  the  following  topics:  complexity  of  skill 
learning,  structure  of  device  training,  instructors, 
attitudes,  assessment  of  training  effectiveness,  and 
maintaining  device  effectiveness. 

The  summary  of  studies  of  life  cycle  cost  and  worth  of 
ownership  covers  the  following  topics:  cost  model 
development,  cost  model  uses,  cost  model  procedures,  and 
worth  of  ownership  assessment. 

Future  research  plans  are  also  outlined  in  this  report. 
Reference  is  made  to  an  abstract  bibliography,  and  an  index 
is  provided  for  the  STRES  technical  reports  on  fidelity, 
instructional  features,  utilization,  and  cost/worth. 
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(f)  Can  vary  the  difficulty  level 

(g)  Can  provide  and  overview 

Psychological  realism,  as  distinct  from  physical  realism, 
should  be  emphasized.  Psychologically  unimportant  details 
of  a  task  or  environment  can  be  left  out  of  simulation  for 
more  effective  learning. 

Despite  the  demonstrated  effectiveness  of  simulation,  this 
problem  remains:  "No  predictive  body  of  knowledge  is 
available  which  will  ensure  the  adequate  design  of 
simulations  for  effective  training.  It  does  not  appear  that 
rigorous  scientific  methods  will  be  applied  to  isolate  and 
test  possible  principles."  Instead  of  a  systematic  approach 
to  isolate  principles,  the  case  study  approach  is 
recommended:  "...we  have  performed  enough  persuasive 

research  through  simulation.  It  appears. .. that  thorough 
case  studies  can  provide  a  way  in  which  information  can  be 
usefully  provided. " 


1.  Author:  Valverde,  Horace  H. 

2.  Title:  A  Review  of  Flight  Simulator  Transfer  of 
Training  Studies 

3.  Source:  Human  Factors  ,  1973,  15(6),  510-523 

4.  Topic  Keywords:  Flight  Simulation  ; 

Transfer  of  Training  ;  Platform  Motion  . 

5.  Short  Summary:  A  review  of  approximately  29  empirical 
and  5  theoretical  studies  in  flight  simulator  effectiveness 
as  measured  by  Transfer  of  Training  is  presented.  The 
review  points  out  disparities  in  findings  and  concludes  by 
identifying  the  chief  confounding  variables:  criteria 
measures,  matching  of  subjects,  instructors,  and 
instructional  sequence. 

6.  Devices: 

a.  Navy  SNJ  Operational  Flight  Trainer 

b.  SNJ  Cycloramic  Link  Trainer 

c.  P-1  (SNJ)  Link  Trainer 

d.  AN-T-18  Link  Trainer 

e.  l-CA-2  SNJ  Link  Trainer 

f.  C-ll  Link  Trainer 

g.  B-47  Link  Simulator 

7.  Institutions: 

a.  Sponsor:  ,  Air  Force  Human  Resources  Laboratory  , 

Advanced  Systems  Division,  Wr ight-Patter son  AFB,  OH 
45433 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review 

1.  Abstract:  This  review  of  transfer-of- training  research 
on  the  effectiveness  of  synthetic  flight  trainers  begins 
with  general  arguments  in  favor  of  the  use  of  simulators  in 
training;  summarizes,  one-by-one,  the  methods  and  findings 
of  approximately  29  empirical  studies;  and  concludes  by 
identifying  the  chief  confounding  variables  in  transfer-of- 
training  experiments. 


General  arguments  in  favor  of  the  use  of  simulators  in 
flight  training,  as  opposed  to  the  use  of  actual  equipment, 
are:  (1)  lower  equipment  costs;  (2)  ease  of  introducing 

malfunctions;  and  (3)  safety. 

The  transf er-of-training  research  which  is  reviewed  focusses 
on  procedures  and  basic  instrument  trainers  during  the 
period  1949-1970  (all  but  5  of  the  studies  are  prior  to 
1966).  Studies  are  categorized  under  three  headings:  Fixed 
Wing;  Motion;  Rotary  Wing  (the  Motion  category  is  actually 
a  sub-category  of  fixed-wing) . 

The  author  observes  in  conclusion,  that  "the  various  studies 
on  transfer  of  training  from  the  flight  simulator  to  the 
actual  aircraft  have  sometimes  produced  contradictory 
results.  The  reason  for  the  disparity  in  findings  may  have 
been  due  to  variables  which  were  not  assessed  in  the 
experiments."  Four  chief  confounding  variables  are 
identified : 

(a)  The  criterion  measure.  Most  performance  measures 
are  "judgemental" — i.e.,  subjective,  instructor- 
scored. 

(b)  Subjects  are  matched  by  paper  and  pencil  tests 
which  may  be  poor  indicators  of  piloting  potential. 

(c)  The  impact  of  instructors  on  simulated  training  is 
virtually  immeasurable  although  suspected  to  be  large. 

(d)  The  instructional  sequence  varies  from  study  to 
study;  its  impact  also  is  virtually  immeasurable  given 
present  data;  a  recent  study  suggests  that  block 
simulator  instruction  is  preferable  to  alternating 
sequence. 
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1.  Author:  Waag,  Wayne  L. 


2.  Title:  Training  Effectiveness  of  Visual  and  Motion 
Simulation 

3.  Source:  AFHRL-TR-7 9-72 ,  January  1981 

4.  Topic  Keywords:  Visual  simulation  ;  Motion  simulation 
;  Flight  simulation  ;  Transfer  of  training  ; 

Flight  training  . 

5.  Short  Summary:  A  review  of  available  studies  of 
transfer  of  training  from  visual  and  motion  simulation  to 
aircraft  reveals  the  general  effectiveness  of  simulator 
training  but  points  to  the  lack  of  experimental  data  which 
would  demonstrate  a  systematic  quantifiable  relationship 
between  simulation  fidelity  and  degree  of  transfer. 

6.  Devices:  Flight  Simulators:  l-CA-2-Link;  PI; 

SNJ  Link;  T-4G;  AS PT/T3 7 ;  2F87F;  2F103NCLT;  ASPT/A-10; 

FFT ;  LAS/WAVS;  SAAC ;  CH047FS;  UH-1FS;  GAT-2 

7.  Institutions: 

0* 

a.  Sponsor:  HQ  Air  Force  Human  Resources  Laboratory  , 
Brooks  Air  Force  Base,  TX  78235 

b.  Performing  Organization:  Operations  Training 
Divbision,  AFHRL ,  Williams  Air  Force  Base,  AZ  85224 

8.  Type  of  Article:  Review 

9.  Abstract:  A  review  of  available  studies  of  transfer  of 
training  from  flight  simulators  to  aircraft  reveals  the 
general  effectiveness  of  simulator  training  but  points  to 
the  lack  of  experimental  data  which  would  demonstrate  a 
systematic  quantifiable  relationship  between  simulation 
fidelity  and  degree  of  transfer.  In  particular,  the 
literature  suggests  that  the  addition  of  visual  system  will 
enhance  the  training  value  of  a  simulator,  whereas  the 
addition  of  a  platform  motion  system  will  have  little 
effect. 

In  this  review,  28  different  studies  are  cited,  over  a 
period  from  1949  to  1980,  investigating  the  transfer  of 
simulation  flight  training  to  real  aircraft.  The  studies 
are  primarily  related  to  visual  flight  simulation  of  as  high 
fidelity  as  possible;  10  of  the  studies  focussed  on  the 
effects  of  motion  simulation.  Flight  tasks  studied 
comprised  landing/takeof f  maneuvers,  formation  flight, 
acrobatics  and  air  combat,  air-to-surface  weapons  delivery, 
and  rotary  wing  flight. 
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The  author  points  out  several  problem  areas  in  the  field  of 
flight  simulation  experimentation: 

(a)  The  foremost  problem  is  that  of  the  required 
fidelity  of  simulation.  Noting  that  many  studies  show 
that  full  fidelity  is  not  requird,  the  author  specified 
the  unanswered  issues  bearing  on  the  fidelity 
requirement ; 

(i)  what  skills  transfer  from  the  simulator  to  the 
aircraf  t? 

(ii)  what  is  the  minimum  necessary  degree  of 
physical  realism? 

(iii)  what  cost  effectiveness  model  could  be  used 
to  compare  cost  of  learning  particular  skills  on 
the  simulator  versus  the  aircraft? 

(b)  Problems  of  Experimental  Design  and  Measurement 
exist  among  various  studies: 

(i)  Variability  of  Research  Objectives 

(ii)  Variability  of  Experimental  Design  and 
Control 

(iii)  Dearth  of  objective  measures  of  proficiency 

(iv)  Variability  in  sample  size  with  an 
unfortunate  tendency  to  smaller  simple  sizes 

(v)  Some  minimal  questions  as  to  what  tasks  under 
study  are  appropriate 

(vi)  "Generalizability ",  i.e.,  that  "there  exist 
no  quantifiable  models  of  visual  and  motion 
simulation  which  enable  testable  hypotheses  to  be 
generated  which  might  subsequently  lead  to  some 
generalizable  findings....  The  research 
community,  in  its  attempt  to  provide  'real  world' 
solutions  for  today's  problems,  has  failed  to 
develop  the  framework  for  obtaining  data  for 
tomorrow's  issues." 

Authors'  Conclusions:  The  literature  suggests  that  the 
addition  of  a  visual  system  will  enhance  the  training  value 
of  the  simulator,  whereas  the  addition  of  a  platform  motion 
system  will  have  little  effect.  However,  there  are  dangers 
in  attempting  to  draw  conclusions  from  diverse  and  often 
unrelated  research  studies.  In  many  cases,  study  goals  are 
different,  and  the  experimental  data  design  and  measurements 
are  different.  Each  of  these  factors  will  have  an  effect 
(usually  unknown)  on  the  study  outcome. 
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1.  Authors:  Wheaton,  George  R. ,  Rose,  Andrew  M. , 

Fingerman,  Paul  W. ,  Korotkin,  Arthur  L. ,  Holding,  Dennis  H. , 
&  Mirabella,  Angelo 

2.  Title:  Evaluation  of  the  Effectiveness  of  Training 
Devices:  Literature  Review  and  Preliminary  Model 

3.  Source:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences  Research  Memorandum  76-6, 
April  1976 

4.  Topic  Keywords:  Transfer  of  Training;  Task  Analysis; 
Fidelity  of  Simulation  ;  Predictive  Model; 

Training  Effectiveness 

5.  Short  Summary:  A  literature  review  and  discussion  of 
theoretical  issues  in  simulation  provides  the  basis  for  a 
preliminary  model  to  predict  training  effectiveness  of 
devices. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  Alexandria,  VA 

b.  Performing  Organization:  same 

8.  Type  of  Article:  Review/Theoretical 

9.  Abstract:  The  technological  capacity  for  building 
training  devices  has  "far  outstripped  knowledge  about  how  to 
design  them."  The  kind  of  model  or  conceptual  framework 
required  to  provide  systematic  guidelines  by  which  to 
predict  training  device  effectiveness  is  lacking.  This 
report  is  the  first  in  a  series  describing  research  aimed  at 
developing  and  validating  a  method  for  predicting  training 
device  effectiveness.  The  report  culminates  in  the 
presentation  of  a  preliminary  model  for  predicting  transfer 
of  training,  considered  one  of  the  most  important,  if  not 
the  most  important,  indicators  of  training  effectiveness. 

The  report  is  divided  into  four  main  sections: 

(1)  Discusses  the  results  of  a  literature  survey; 

(2)  Outlines  major  psychological  theories  of  Transfer 
of  Training; 

(3)  Summarizes  the  empirical  literature  on  learning  and 
transfer ; 

(4)  Describes  the  predictive  model. 

The  first  section,  in  reviewing  the  available  literature, 
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describes  a  number  of  models  which  are  seen  to  fall  under 
two  general  headings:  task  analytic,  which  focus  on 
learning  acquisition  rather  than  transfer,  providing 
prescription  rather  than  prediction;  micro-analytic,  which 
focus  on  comparing  specific  stimuli  and  responses,  providing 
either  prescription  or  prediction.  The  review  concludes 
that  no  existing  models  are  entirely  adequate  for 
prediction , 

The  second  section  discusses  theoretical  positions  and 
issues,  specifically  similarity  theories,  mediation 
theories,  the  nature  of  negative  transfer,  fidelity,  and  the 
roles  of  amount  and  stage  of  training  in  transfer.  The 
similarity  theory  of  transfer  appears  stronger,  but  "...  it 
must  be  recognized  that  current  similarity  models  seem 
incapable  of  accommodating  all  of  the  gradients  of 
similarity  which  are  possible...  These  similarity  relations 
by  no  means  exhaust  the  list  of  relevant  and  powerful 
variables.  Similarity  may  be  a  necessary  condition  for 
transfer  to  occur,  but  it  surely  is  not  sufficient . " 

The  third  section  summarizes  65  studies  in  the  empirical 
literature.  It  begins  by  identifying  and  listing  the 
substantive  variables,  indexed  to  relevant  documents.  The 
variables  are  classified  under  three  main  headings: 

(1)  "Training"  variables,  related  to  the  efficiency 
with  which  any  task  could  be  trained;  e.g.,  amount  of 
practice,  knowledge  of  results,  etc.; 

(2)  "Device"  variables,  related  to  the  appropriateness 
of  the  device  to  the  task  to  be  learned,  e.g., 
fidelity; 

(3)  "Task"  variables,  related  to  the  task  itself,  e.g., 
difficulty,  duration,  etc. 

The  section  goes  on  to  discuss  briefly  each  of  the  variables 
in  turn.  It  is  noted  that  the  fidelity  variables  have  drawn 
the  most  attention  in  research:  "Fidelity  has  been  the 
cornerstone  of  several  device  effectiveness  models  and  the 
basis  of  most  transfer  of  training  theories."  The  question 
is  considered,  of  "why  the  data  are  so  unconvincing  in 
support  of  fidelity  as  a  direct  predictor  of  training 
ef feciveness, "  and  it  is  observed  that  the  effects  of 
fidelity  on  transfer  are  large:  however,  these  effects  are 
modified...  by  other  considerations,"  such  as  task 
difficulty,  amount  of  practice,  etc.  This  section  concludes 
with  a  rationale  for  the  synthesis  of  general  training 
principles,  and  a  list  of  principles  inferred  from  the 
literature  is  provided. 

The  final  section  of  the  paper  describes  the  preliminary 
model  which  attempts  to  generalize  and  integrate  the 
concepts  introduced  earlier  in  the  paper.  This  is  done  with 
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a  recognition  that  "...  the  literature  consists  mostly  of 
studies  with  such  different  approaches,  measures,  controls, 
and  variables  that  it  is  impossible  to  reconcile,  let  alone 
assimilate,  all  of  their  findings...  there  is  little  which 
can  be  systematized  into  coherent  principles  or  laws." 

The  tentative  model  which  emerges  is  "basically  a  training 
content  by  training  process  model."  It  organizes  content  by 
task  description  and  relevant  behavioral  categories;  it 
organizes  process  by  "appropriateness"  and  "efficiency",  and 
finally  by  their  product,  "effectiveness". 
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1.  Author:  Whiteside,  George  A. 

2.  Title:  Aircraft  Simulator  Data  Requirements  Study: 
Executive  Summary 

3.  Source:  Final  Report,  ASD-TR-77-25  Vol.  I,  January 
1977 

4.  Topic  Keywords:  Flight  Simulation  ; 

Computer-Aided  Design  . 

5.  Short  Summary:  A  survey  of  flight  simulator 
manufacturers,  simulator  acquisition  activities,  and 
aircraft  manuf acturers  yields  criteria  of  a  General 
Requirement  for  supplying  data  for  simulator  development. 
Such  a  requirement  would  constitute  an  up-to-date  standard 
for  specifying  the  data  needed  by  simulator  manufactures, 
including  makers  of  digital  computer-driven  flight 
simulators. 

6.  Devices:  None  discussed 

7.  Institutions: 

a.  Sponsor:  Aeronautical  Systems  Division, 
Wright-Patterson  AFB ,  OH  45433 

b.  Performing  Organization: 

Systems  Research  Laboratories,  Inc. ,  2800  Indian 
Ripple  Road,  Dayton,  OH  45440 

8.  Type  of  Article:  Conceptual/Methodological 

9.  Abstract:  Impediments  to  the  flow  of  data  on  new 
aircraft  to  simulator  manufacturers,  and  suggestions  for 
overcoming  them,  constitute  the  subject  matter  of  this 
report.  A  survey  of  flight  simulator  manufacturers, 
simulator  acquisition  activities,  and  aircraft  manufacturers 
was  performed  to  determine  a  General  Requirement  for 
supplying  the  necessary  data  for  simulator  development  on  a 
schedule  such  that  "the  simulator  can  be  on-site,  ready  for 
training  before  the  first  production  aircraft  is  delivered 
to  the  operating  command." 

Discussion  of  the  survey  touches  on  the  following  topics  and 
problems  in  the  acquisition  of  data  for  simulation: 
acceptance  test  procedure;  malfunction  and  abnormal 
operations;  barriers  to  communication  between  simulator 
manufacturer  and  aircraft  design  engineers;  timing  of 
physical  data,  aerodynamic  data  estimate,  system  data 
estimates,  "hot  bench"  tests,  and  dynamic  tests; 
intelligence  data;  tactical  program  tapes  and  duplication 
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of  aircraft  central  computers;  enforcement  of  the 
data-acquisition  schedule;  procurement  of  crew  station 
parts  from  the  aircraft  manufacturer;  project  pilots  and 
project  crew  members;  and  role  of  instructor  pilots. 

A  number  of  specific  recommendations  are  made  to  facilitate 
the  supplying  of  useable  data  to  the  simulator  manufacturer. 
The  primary  recommendations  are,  that  the  initial  contract 
for  aircraft  development  should  require  delivery  of  the  data 
needed  to  develop  a  full  mission  simulator,  and  that  the 
activity  should  follow  up  on  these  requirements  to  ensure  a 
timely  delivery  of  accurate  information.  The  Air  Force 
Flight  Test  Center  must  also  be  tasked  to  provide  handling 
qualities  and  performance  parameters  to  the  simulator 
manufacturer  as  early  as  possible  in  the  test  program;  the 
AFFTC  should  also  assign  a  qualified  test  pilot  current  in 
the  aircraft,  and  a  flight  test  engineer  to  assist  in  the 
development  of  the  simulator. 
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2.  Title:  Environmental  Requirements  for  Simulated 
Helicopter/VTOL  Operations  from  Small  Ships  and  Carriers 

3.  Source:  Technical  Memo,  Naval  Air  Test  Center 
TM-78-2RW,  April  1978 

4.  Topic  Keywords:  Rotary  Wing  Flight  Simulation  ; 
Helicopter  Simulation  Training  ;  Weapon  System  Trainer  ; 
Shipboard  Environment  ;  Training  Effectiveness; 

Vertical  Take-Off  Landing  Operations  Training  ; 

Visual  Landing  Aids  . 

5.  Short  Summary:  The  simulation  of  helicopter  flight 
operation  via  high-fidelity  state-of-the-art  devices  improve 
the  safety  and  economics  of  training. 

6.  Devices:  2F106  SH-2F  Weapons  System  Trainer 

7.  Institutions: 

a.  Sponsor:  Naval  Air  Systems  Command,  Department  of 
the  Navy,  Washington,  DC  20361 

b.  Performing  Organization: 

Rotary  Wing  Aircraft  Test 

Directorate,  Naval  Air  Test  Center,  Patuxent  River,  MD 
20670 

8.  Type  of  Article:  Conceptual/Review 

9.  Abstract:  Helicopter/VTOL  operations  from  ships  have 
characteristics  which  differ  greatly  from  those  of 
fixed-wing  flying  tasks  and  appear  to  demand  a  high  degree 
of  realism  in  training  simulation.  This  report  discusses 
the  demands  that  this  unique  flight  environment  makes  of 
training  simulators,  and  describes  developing 
state-of-the-art  technology  for  meeting  these  demands, 
particularly  as  exemplified  by  Device  2F106,  the  SH-2F 
Weapons  System  Trainer  (WST) . 

The  demands  of  simulating  helicopter/VTOL  operations  from 
ships  are: 

(a)  Low  altitude,  low  airspeed  flight  regime  is 
particularly  limited  by  visual  system  lag  time. 

Maximum  tolerances  are  a  few  hundred  milliseconds; 

(b)  Response  in  all  visual  axes  is  critical;  yaw 
response  is  much  more  demanding  than  in  a  fixed  wing 
aircraft; 

(c)  Field-of- view  in  simulation  is  probably  more 
demanding  than  in  fixed-wing  aircraft  simulation; 
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(d)  Occultation,  as  of  the  sea  horizon  by  a  ships's 
super-  structure,  is  mandated,  as  horizon  image  when  in 
view  appears  to  take  precedence  over  other  scene 
content ; 

(e)  "A  full  6  degrees  of  freedom  motion  system  appears 
to  be...  a  necessity  to  VTOL  trainers.... 

Significantly  improved  pilot  performance  has  been  noted 
with  the  inclusion  of  the  system."; 

(f)  Ship  motion  models  are  required; 

(7)  Turbulence,  relative  wind,  ground  aerodynamic 
effect,  and  sound,  are  all  highly  important  workload 
and/or  cueing  elements  for  the  helicopter  pilot. 

State-of-the-art  calligraphic  visual  systems  provide  highly 
promising  visual  simulation  capabilities,  including 
occultation,  and  dynamic  focussing  and  defocussing  of  scene 
elements,  as  new  capabilities. 

Another  technological  improvement  is  in  the  reduction  of  lag 
time  in  the  visual  system.  In  VITAL  IV  image  generation, 
lag  has  been  demonstrated  to  be  less  than  50  milliseconds. 

The  authors  conclude  by  reiterating  the  need  for  high 
fidelity  in  training  simulation  in  this  application,  and  by 
emphasizing  the  importance  of  new  technologies  in  meeting 
this  need. 
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